hosted engine setup failed for 10 minutes delay.. engine seems alive
Hello, trying to deploy a self hosted engine on an Intel NUC6i5SYB with CentOS 7.2 using oVirt 3.6.5 and appliance (picked up rpm is ovirt-engine-appliance-3.6-20160420.1.el7.centos.noarch) Near the end of the command hosted-engine --deploy I get ... |- [ INFO ] Initializing PostgreSQL |- [ INFO ] Creating PostgreSQL 'engine' database |- [ INFO ] Configuring PostgreSQL |- [ INFO ] Creating/refreshing Engine database schema |- [ INFO ] Creating/refreshing Engine 'internal' domain database schema [ ERROR ] Engine setup got stuck on the appliance [ ERROR ] Failed to execute stage 'Closing up': Engine setup is stalled on the appliance since 600 seconds ago. Please check its log on the appliance. [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue, fix and redeploy On host log I indeed see the 10 minutes timeout: 2016-04-30 19:56:52 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:219 DIALOG:SEND |- [ INFO ] Creating/refreshing Engine 'internal' domain database schema 2016-04-30 20:06:53 ERROR otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:140 Engine setup got stuck on the appliance On engine I don't see any particular problem but a ten minutes delay in its log: 2016-04-30 17:56:57 DEBUG otopi.context context.dumpEnvironment:514 ENVIRONMENT DUMP - END 2016-04-30 17:56:57 DEBUG otopi.context context._executeMethod:142 Stage misc METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc.Plugin._setupAdminPassword 2016-04-30 17:56:57 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z'), executable='None', cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH': '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**', 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/', 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE': '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log', 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly', 'OTOPI_EXECDIR': '/'} 2016-04-30 18:07:06 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:878 execute-result: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z'), rc=0 and its last lines are: 2016-04-30 18:07:06 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.execute:936 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z') stdout: updating user admin... user updated successfully 2016-04-30 18:07:06 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.execute:941 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z') stderr: 2016-04-30 18:07:06 DEBUG otopi.context context._executeMethod:142 Stage misc METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine.pki.ca.Plugin._miscUpgrade 2016-04-30 18:07:06 INFO otopi.plugins.ovirt_engine_setup.ovirt_engine.pki.ca ca._miscUpgrade:510 Upgrading CA Full logs of host and engine here: https://drive.google.com/file/d/0BwoPbcrMv8mvQm9jeDhpZEdRUjg/view?usp=sharin... I can connect via vnc to the engine and see 277 tables in engine database (277 rows in output of "\d" command) Can anyone tell me if I can follow up without starting from scratch and how in case? Also understand the reason of this delay, as the NUC is a physical host with 32Gb of ram and SSD disks and should be quite fast... faster than a VM non my laptop where I had no problems in similar setup... As a last question how to clean up things in case I have to start from scratch. I can leave the situation as it is in the moment, so I can work on the live environment before power off.... Thanks in advance, Gianluca
On Sat, Apr 30, 2016 at 10:59 PM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
Hello, trying to deploy a self hosted engine on an Intel NUC6i5SYB with CentOS 7.2 using oVirt 3.6.5 and appliance (picked up rpm is ovirt-engine-appliance-3.6-20160420.1.el7.centos.noarch)
Near the end of the command hosted-engine --deploy
I get ... |- [ INFO ] Initializing PostgreSQL |- [ INFO ] Creating PostgreSQL 'engine' database |- [ INFO ] Configuring PostgreSQL |- [ INFO ] Creating/refreshing Engine database schema |- [ INFO ] Creating/refreshing Engine 'internal' domain database schema [ ERROR ] Engine setup got stuck on the appliance [ ERROR ] Failed to execute stage 'Closing up': Engine setup is stalled on the appliance since 600 seconds ago. Please check its log on the appliance. [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue, fix and redeploy
On host log I indeed see the 10 minutes timeout:
2016-04-30 19:56:52 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:219 DIALOG:SEND |- [ INFO ] Creating/refreshing Engine 'internal' domain database schema 2016-04-30 20:06:53 ERROR otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:140 Engine setup got stuck on the appliance
On engine I don't see any particular problem but a ten minutes delay in its log:
2016-04-30 17:56:57 DEBUG otopi.context context.dumpEnvironment:514 ENVIRONMENT DUMP - END 2016-04-30 17:56:57 DEBUG otopi.context context._executeMethod:142 Stage misc METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc.Plugin._setupAdminPassword 2016-04-30 17:56:57 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z'), executable='None', cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH': '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**', 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/', 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE': '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log', 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly', 'OTOPI_EXECDIR': '/'} 2016-04-30 18:07:06 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:878 execute-result: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z'), rc=0
and its last lines are:
2016-04-30 18:07:06 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.execute:936 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z') stdout: updating user admin... user updated successfully
hosted-engine-setup creates a fresh VM and inject a cloud-init script to configure it and execute there engine-setup to configure the engine as needed. Since engine-setup is running on the engine VM triggered by cloud-init, hosted-engine-setup has no way to really control its process status so we simply gather its output with a timeout of 10 minutes between each single output line. In nothing happens within 10 minutes (the value is easily customizable), hosted-engine-setup thinks that engine-setup is stuck. So the issue we have to understood is why this simple command took more than 10 minutes in your env: 2016-04-30 17:56:57 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z'), executable='None', cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH': '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**', 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/', 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE': '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log', 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly', 'OTOPI_EXECDIR': '/'} Can you please check the entropy value on your host? cat /proc/sys/kernel/random/entropy_avail
2016-04-30 18:07:06 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.execute:941 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z') stderr:
2016-04-30 18:07:06 DEBUG otopi.context context._executeMethod:142 Stage misc METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine.pki.ca.Plugin._miscUpgrade 2016-04-30 18:07:06 INFO otopi.plugins.ovirt_engine_setup.ovirt_engine.pki.ca ca._miscUpgrade:510 Upgrading CA
Full logs of host and engine here: https://drive.google.com/file/d/0BwoPbcrMv8mvQm9jeDhpZEdRUjg/view?usp=sharin...
I can connect via vnc to the engine and see 277 tables in engine database (277 rows in output of "\d" command)
Can anyone tell me if I can follow up without starting from scratch and how in case? Also understand the reason of this delay, as the NUC is a physical host with 32Gb of ram and SSD disks and should be quite fast... faster than a VM non my laptop where I had no problems in similar setup...
As a last question how to clean up things in case I have to start from scratch.
I'd recommend to redeploy from scratch instead of trying fixing it but, before that, we need to understand the root issue.
I can leave the situation as it is in the moment, so I can work on the live environment before power off....
Thanks in advance, Gianluca
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Mon, May 2, 2016 at 9:58 AM, Simone Tiraboschi wrote:
hosted-engine-setup creates a fresh VM and inject a cloud-init script to configure it and execute there engine-setup to configure the engine as needed. Since engine-setup is running on the engine VM triggered by cloud-init, hosted-engine-setup has no way to really control its process status so we simply gather its output with a timeout of 10 minutes between each single output line. In nothing happens within 10 minutes (the value is easily customizable), hosted-engine-setup thinks that engine-setup is stuck.
How can one customize the pre-set timeout? Could it be better to ask the user at the end of timeout if he/she wants to wait again, instead of directly fail?
So the issue we have to understood is why this simple command took more than 10 minutes in your env: 2016-04-30 17:56:57 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z'), executable='None', cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH': '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**', 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/', 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE': '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log', 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly', 'OTOPI_EXECDIR': '/'}
It seemed quite strange to me too (see below further info on this)
Can you please check the entropy value on your host? cat /proc/sys/kernel/random/entropy_avail
I have not at hand now the server. I'll check soon and report Do you mean entropy of the physical server that will operate as hypervisor?
As a last question how to clean up things in case I have to start from scratch.
I'd recommend to redeploy from scratch instead of trying fixing it but, before that, we need to understand the root issue.
So, trying restart the setup with generated answer file I got: 1) if VM still powered on, an error about this condition 2) if VM powered down, an error abut storage domain already in place and restart not supported in this condition. I was able to continue with these steps: a) remove what inside the partially setup self hosted engine storage domain rm -rf /SHE_DOMAIN/* cd SHE_DOMAIN mklost+found b) reboot the hypervisor c) stop vdsmd d) start the setup again with the answer file It seems all went well and this time strangely the step that took more than 10 minutes before lasted less than 2 seconds.... I was then able to deploy storage and iso domains without problems and self hosted engine domain correctly detected and imported too. Created two CentOS VMs without problems (6.7 and 7.2). See below the full output of deploy command [root@ovirt01 ~]# hosted-engine --deploy --config-append=/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf [ INFO ] Stage: Initializing [ INFO ] Generating a temporary VNC password. [ INFO ] Stage: Environment setup Configuration files: ['/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf'] Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20160501014326-8frbxk.log Version: otopi-1.4.1 (otopi-1.4.1-1.el7.centos) [ INFO ] Hardware supports virtualization [ INFO ] Bridge ovirtmgmt already created [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup [ INFO ] Stage: Environment customization --== STORAGE CONFIGURATION ==-- During customization use CTRL-D to abort. [ INFO ] Installing on first host --== SYSTEM CONFIGURATION ==-- --== NETWORK CONFIGURATION ==-- --== VM CONFIGURATION ==-- [ INFO ] Checking OVF archive content (could take a few minutes depending on archive size) [ INFO ] Checking OVF XML content (could take a few minutes depending on archive size) [WARNING] OVF does not contain a valid image description, using default. Enter root password that will be used for the engine appliance (leave it empty to skip): Confirm appliance root password: [ INFO ] The engine VM will be configured to use 192.168.1.212/24 The following CPU types are supported by this host: - model_Broadwell-noTSX: Intel Broadwell-noTSX Family - model_Haswell-noTSX: Intel Haswell-noTSX Family - model_SandyBridge: Intel SandyBridge Family - model_Westmere: Intel Westmere Family - model_Nehalem: Intel Nehalem Family - model_Penryn: Intel Penryn Family - model_Conroe: Intel Conroe Family [WARNING] Minimum requirements for CPUs not met [WARNING] Minimum requirements for disk size not met --== HOSTED ENGINE CONFIGURATION ==-- Enter the name which will be used to identify this host inside the Administrator Portal [hosted_engine_1]: Enter 'admin@internal' user password that will be used for accessing the Administrator Portal: Confirm 'admin@internal' user password: [ INFO ] Stage: Setup validation --== CONFIGURATION PREVIEW ==-- Bridge interface : eth0 Engine FQDN : ovirt.lutwyn.org Bridge name : ovirtmgmt Host address : ovirt01.lutwyn.org SSH daemon port : 22 Firewall manager : iptables Gateway address : 192.168.1.1 Host name for web application : hosted_engine_1 Host ID : 1 Image size GB : 10 GlusterFS Share Name : hosted_engine_glusterfs GlusterFS Brick Provisioning : False Storage connection : ovirt01.lutwyn.org: /SHE_DOMAIN Console type : vnc Memory size MB : 8192 MAC address : 00:16:3e:2e:93:15 Boot type : disk Number of CPUs : 1 OVF archive (for disk boot) : /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-3.6-20160420.1.el7.centos.ova Restart engine VM after engine-setup: True CPU Type : model_Broadwell-noTSX [ INFO ] Stage: Transaction setup [ INFO ] Stage: Misc configuration [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Configuring libvirt [ INFO ] Configuring VDSM [ INFO ] Starting vdsmd [ INFO ] Waiting for VDSM hardware info [ INFO ] Creating Storage Domain [ INFO ] Creating Storage Pool [ INFO ] Connecting Storage Pool [ INFO ] Verifying sanlock lockspace initialization [ INFO ] Creating Image for 'hosted-engine.lockspace' ... [ INFO ] Image for 'hosted-engine.lockspace' created successfully [ INFO ] Creating Image for 'hosted-engine.metadata' ... [ INFO ] Image for 'hosted-engine.metadata' created successfully [ INFO ] Creating VM Image [ INFO ] Extracting disk image from OVF archive (could take a few minutes depending on archive size) [ INFO ] Validating pre-allocated volume size [ INFO ] Uploading volume to data domain (could take a few minutes depending on archive size) [ INFO ] Image successfully imported from OVF [ INFO ] Destroying Storage Pool [ INFO ] Start monitoring domain [ INFO ] Configuring VM [ INFO ] Updating hosted-engine configuration [ INFO ] Stage: Transaction commit [ INFO ] Stage: Closing up [ INFO ] Creating VM You can now connect to the VM with the following command: /bin/remote-viewer vnc://localhost:5900 Use temporary password "4699ouyH" to connect to vnc console. Please note that in order to use remote-viewer you need to be able to run graphical applications. This means that if you are using ssh you have to supply the -Y flag (enables trusted X11 forwarding). Otherwise you can run the command from a terminal in your preferred desktop environment. If you cannot run graphical applications you can connect to the graphic console from another host or connect to the serial console using the following command: socat UNIX-CONNECT:/var/run/ovirt-vmconsole-console/4dae6016-ff01-4a1f-87ea-09f049d0e58c.sock,user=ovirt-vmconsole STDIO,raw,echo=0,escape=1 Please ensure that your Guest OS is properly configured to support serial console according to your distro documentation. Follow http://www.ovirt.org/Serial_Console_Setup#I_need_to_access_the_console_the_o... for more info. If you need to reboot the VM you will need to start it manually using the command: hosted-engine --vm-start You can then set a temporary password using the command: hosted-engine --add-console-password [ INFO ] Running engine-setup on the appliance |- [ INFO ] Stage: Initializing |- [ INFO ] Stage: Environment setup |- Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/root/ovirt-engine-answers', '/root/heanswers.conf'] |- Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20160430234733-h0rns4.log |- Version: otopi-1.4.1 (otopi-1.4.1-1.el7.centos) |- [ INFO ] Stage: Environment packages setup |- [ INFO ] Stage: Programs detection |- [ INFO ] Stage: Environment setup |- [ INFO ] Stage: Environment customization |- |- --== PRODUCT OPTIONS ==-- |- |- |- --== PACKAGES ==-- |- |- |- --== ALL IN ONE CONFIGURATION ==-- |- |- |- --== NETWORK CONFIGURATION ==-- |- |- [ INFO ] firewalld will be configured as firewall manager. |- |- --== DATABASE CONFIGURATION ==-- |- |- |- --== OVIRT ENGINE CONFIGURATION ==-- |- |- |- --== STORAGE CONFIGURATION ==-- |- |- |- --== PKI CONFIGURATION ==-- |- |- |- --== APACHE CONFIGURATION ==-- |- |- |- --== SYSTEM CONFIGURATION ==-- |- |- |- --== MISC CONFIGURATION ==-- |- |- |- --== END OF CONFIGURATION ==-- |- |- [ INFO ] Stage: Setup validation |- [WARNING] Cannot validate host name settings, reason: resolved host does not match any of the local addresses |- [WARNING] Less than 16384MB of memory is available |- |- --== CONFIGURATION PREVIEW ==-- |- |- Application mode : virt |- Default SAN wipe after delete : False |- Firewall manager : firewalld |- Update Firewall : True |- Host FQDN : ovirt.lutwyn.org |- Engine database secured connection : False |- Engine database host : localhost |- Engine database user name : engine |- Engine database name : engine |- Engine database port : 5432 |- Engine database host name validation : False |- Engine installation : True |- PKI organization : lutwyn.org |- Configure local Engine database : True |- Set application as default page : True |- Configure Apache SSL : True |- Configure VMConsole Proxy : True |- Engine Host FQDN : ovirt.lutwyn.org |- Configure WebSocket Proxy : True |- [ INFO ] Stage: Transaction setup |- [ INFO ] Stopping engine service |- [ INFO ] Stopping ovirt-fence-kdump-listener service |- [ INFO ] Stopping websocket-proxy service |- [ INFO ] Stage: Misc configuration |- [ INFO ] Stage: Package installation |- [ INFO ] Stage: Misc configuration |- [ INFO ] Initializing PostgreSQL |- [ INFO ] Creating PostgreSQL 'engine' database |- [ INFO ] Configuring PostgreSQL |- [ INFO ] Creating/refreshing Engine database schema |- [ INFO ] Creating/refreshing Engine 'internal' domain database schema |- [ INFO ] Upgrading CA |- [ INFO ] Creating CA |- [ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts |- [ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts |- [ INFO ] Configuring WebSocket Proxy |- [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' |- [ INFO ] Stage: Transaction commit |- [ INFO ] Stage: Closing up |- |- --== SUMMARY ==-- |- |- [WARNING] Less than 16384MB of memory is available |- SSH fingerprint: 13:dc:47:b9:38:66:15:eb:83:e8:ed:6f:71:cb:d4:ea |- Internal CA D8:4B:4C:74:45:EB:87:C4:8F:FE:DE:02:DF:99:5E:CB:AA:1B:D84 |- Note! If you want to gather statistical information you can install Reports and/or DWH: |- http://www.ovirt.org/Ovirt_DWH |- http://www.ovirt.org/Ovirt_Reports |- Web access is enabled at: |- http://ovirt.lutwyn.org:80/ovirt-engine |- https://ovirt.lutwyn.org:443/ovirt-engine |- Please use the user 'admin@internal' and password specified in order to login |- |- --== END OF SUMMARY ==-- |- |- [ INFO ] Starting engine service |- [ INFO ] Restarting httpd |- [ INFO ] Restarting ovirt-vmconsole proxy service |- [ INFO ] Stage: Clean up |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20160430234733-h0rns4.log |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20160430234907-setup.conf' |- [ INFO ] Stage: Pre-termination |- [ INFO ] Stage: Termination |- [ INFO ] Execution of setup completed successfully |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS [ INFO ] Engine-setup successfully completed [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Acquiring internal CA cert from the engine [ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct: [ INFO ] Issuer: C=US, O=lutwyn.org, CN=ovirt.lutwyn.org.36587, Subject: C=US, O=lutwyn.org, CN=ovirt.lutwyn.org.36587, Fingerprint (SHA-1): D84B4C7445EB87C48FFEDE02DF995ECBAA1BD84 [ INFO ] Connecting to the Engine [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] The VDSM Host is now operational [ INFO ] Saving hosted-engine configuration on the shared storage domain [ INFO ] Shutting down the engine VM [ INFO ] Enabling and starting HA services [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20160501015145.conf' [ INFO ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ INFO ] Hosted Engine successfully set up [root@ovirt01 ~]#
On Mon, May 2, 2016 at 11:48 AM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, May 2, 2016 at 9:58 AM, Simone Tiraboschi wrote:
hosted-engine-setup creates a fresh VM and inject a cloud-init script to configure it and execute there engine-setup to configure the engine as needed. Since engine-setup is running on the engine VM triggered by cloud-init, hosted-engine-setup has no way to really control its process status so we simply gather its output with a timeout of 10 minutes between each single output line. In nothing happens within 10 minutes (the value is easily customizable), hosted-engine-setup thinks that engine-setup is stuck.
How can one customize the pre-set timeout? Could it be better to ask the user at the end of timeout if he/she wants to wait again, instead of directly fail?
Perhaps, can you please open a bz?
So the issue we have to understood is why this simple command took more than 10 minutes in your env: 2016-04-30 17:56:57 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z'), executable='None', cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH': '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**', 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/', 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE':
'/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log', 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly', 'OTOPI_EXECDIR': '/'}
It seemed quite strange to me too (see below further info on this)
Can you please check the entropy value on your host? cat /proc/sys/kernel/random/entropy_avail
I have not at hand now the server. I'll check soon and report Do you mean entropy of the physical server that will operate as hypervisor?
That's a good question. Simone - do you know if we start the guest with virtio-rng? This is another case of [1], perhaps we should reopen it. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1319827
As a last question how to clean up things in case I have to start from scratch.
I'd recommend to redeploy from scratch instead of trying fixing it but, before that, we need to understand the root issue.
So, trying restart the setup with generated answer file I got: 1) if VM still powered on, an error about this condition 2) if VM powered down, an error abut storage domain already in place and restart not supported in this condition.
I was able to continue with these steps:
a) remove what inside the partially setup self hosted engine storage domain rm -rf /SHE_DOMAIN/* cd SHE_DOMAIN mklost+found
b) reboot the hypervisor
c) stop vdsmd
d) start the setup again with the answer file It seems all went well and this time strangely the step that took more than 10 minutes before lasted less than 2 seconds....
I was then able to deploy storage and iso domains without problems and self hosted engine domain correctly detected and imported too. Created two CentOS VMs without problems (6.7 and 7.2).
See below the full output of deploy command
[root@ovirt01 ~]# hosted-engine --deploy --config-append=/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf [ INFO ] Stage: Initializing [ INFO ] Generating a temporary VNC password. [ INFO ] Stage: Environment setup Configuration files: ['/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf'] Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20160501014326-8frbxk.log Version: otopi-1.4.1 (otopi-1.4.1-1.el7.centos) [ INFO ] Hardware supports virtualization [ INFO ] Bridge ovirtmgmt already created [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup [ INFO ] Stage: Environment customization
--== STORAGE CONFIGURATION ==--
During customization use CTRL-D to abort. [ INFO ] Installing on first host
--== SYSTEM CONFIGURATION ==--
--== NETWORK CONFIGURATION ==--
--== VM CONFIGURATION ==--
[ INFO ] Checking OVF archive content (could take a few minutes depending on archive size) [ INFO ] Checking OVF XML content (could take a few minutes depending on archive size) [WARNING] OVF does not contain a valid image description, using default. Enter root password that will be used for the engine appliance (leave it empty to skip): Confirm appliance root password: [ INFO ] The engine VM will be configured to use 192.168.1.212/24 The following CPU types are supported by this host: - model_Broadwell-noTSX: Intel Broadwell-noTSX Family - model_Haswell-noTSX: Intel Haswell-noTSX Family - model_SandyBridge: Intel SandyBridge Family - model_Westmere: Intel Westmere Family - model_Nehalem: Intel Nehalem Family - model_Penryn: Intel Penryn Family - model_Conroe: Intel Conroe Family [WARNING] Minimum requirements for CPUs not met [WARNING] Minimum requirements for disk size not met
--== HOSTED ENGINE CONFIGURATION ==--
Enter the name which will be used to identify this host inside the Administrator Portal [hosted_engine_1]: Enter 'admin@internal' user password that will be used for accessing the Administrator Portal: Confirm 'admin@internal' user password: [ INFO ] Stage: Setup validation
--== CONFIGURATION PREVIEW ==--
Bridge interface : eth0 Engine FQDN : ovirt.lutwyn.org Bridge name : ovirtmgmt Host address : ovirt01.lutwyn.org SSH daemon port : 22 Firewall manager : iptables Gateway address : 192.168.1.1 Host name for web application : hosted_engine_1 Host ID : 1 Image size GB : 10 GlusterFS Share Name : hosted_engine_glusterfs GlusterFS Brick Provisioning : False Storage connection : ovirt01.lutwyn.org:/SHE_DOMAIN Console type : vnc Memory size MB : 8192 MAC address : 00:16:3e:2e:93:15 Boot type : disk Number of CPUs : 1 OVF archive (for disk boot) : /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-3.6-20160420.1.el7.centos.ova Restart engine VM after engine-setup: True CPU Type : model_Broadwell-noTSX [ INFO ] Stage: Transaction setup [ INFO ] Stage: Misc configuration [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Configuring libvirt [ INFO ] Configuring VDSM [ INFO ] Starting vdsmd [ INFO ] Waiting for VDSM hardware info [ INFO ] Creating Storage Domain [ INFO ] Creating Storage Pool [ INFO ] Connecting Storage Pool [ INFO ] Verifying sanlock lockspace initialization [ INFO ] Creating Image for 'hosted-engine.lockspace' ... [ INFO ] Image for 'hosted-engine.lockspace' created successfully [ INFO ] Creating Image for 'hosted-engine.metadata' ... [ INFO ] Image for 'hosted-engine.metadata' created successfully [ INFO ] Creating VM Image [ INFO ] Extracting disk image from OVF archive (could take a few minutes depending on archive size) [ INFO ] Validating pre-allocated volume size [ INFO ] Uploading volume to data domain (could take a few minutes depending on archive size) [ INFO ] Image successfully imported from OVF [ INFO ] Destroying Storage Pool [ INFO ] Start monitoring domain [ INFO ] Configuring VM [ INFO ] Updating hosted-engine configuration [ INFO ] Stage: Transaction commit [ INFO ] Stage: Closing up [ INFO ] Creating VM You can now connect to the VM with the following command: /bin/remote-viewer vnc://localhost:5900 Use temporary password "4699ouyH" to connect to vnc console. Please note that in order to use remote-viewer you need to be able to run graphical applications. This means that if you are using ssh you have to supply the -Y flag (enables trusted X11 forwarding). Otherwise you can run the command from a terminal in your preferred desktop environment. If you cannot run graphical applications you can connect to the graphic console from another host or connect to the serial console using the following command: socat UNIX-CONNECT:/var/run/ovirt-vmconsole-console/4dae6016-ff01-4a1f-87ea-09f049d0e58c.sock,user=ovirt-vmconsole STDIO,raw,echo=0,escape=1 Please ensure that your Guest OS is properly configured to support serial console according to your distro documentation. Follow http://www.ovirt.org/Serial_Console_Setup#I_need_to_access_the_console_the_o... for more info. If you need to reboot the VM you will need to start it manually using the command: hosted-engine --vm-start You can then set a temporary password using the command: hosted-engine --add-console-password [ INFO ] Running engine-setup on the appliance |- [ INFO ] Stage: Initializing |- [ INFO ] Stage: Environment setup |- Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/root/ovirt-engine-answers', '/root/heanswers.conf'] |- Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20160430234733-h0rns4.log |- Version: otopi-1.4.1 (otopi-1.4.1-1.el7.centos) |- [ INFO ] Stage: Environment packages setup |- [ INFO ] Stage: Programs detection |- [ INFO ] Stage: Environment setup |- [ INFO ] Stage: Environment customization |- |- --== PRODUCT OPTIONS ==-- |- |- |- --== PACKAGES ==-- |- |- |- --== ALL IN ONE CONFIGURATION ==-- |- |- |- --== NETWORK CONFIGURATION ==-- |- |- [ INFO ] firewalld will be configured as firewall manager. |- |- --== DATABASE CONFIGURATION ==-- |- |- |- --== OVIRT ENGINE CONFIGURATION ==-- |- |- |- --== STORAGE CONFIGURATION ==-- |- |- |- --== PKI CONFIGURATION ==-- |- |- |- --== APACHE CONFIGURATION ==-- |- |- |- --== SYSTEM CONFIGURATION ==-- |- |- |- --== MISC CONFIGURATION ==-- |- |- |- --== END OF CONFIGURATION ==-- |- |- [ INFO ] Stage: Setup validation |- [WARNING] Cannot validate host name settings, reason: resolved host does not match any of the local addresses |- [WARNING] Less than 16384MB of memory is available |- |- --== CONFIGURATION PREVIEW ==-- |- |- Application mode : virt |- Default SAN wipe after delete : False |- Firewall manager : firewalld |- Update Firewall : True |- Host FQDN : ovirt.lutwyn.org |- Engine database secured connection : False |- Engine database host : localhost |- Engine database user name : engine |- Engine database name : engine |- Engine database port : 5432 |- Engine database host name validation : False |- Engine installation : True |- PKI organization : lutwyn.org |- Configure local Engine database : True |- Set application as default page : True |- Configure Apache SSL : True |- Configure VMConsole Proxy : True |- Engine Host FQDN : ovirt.lutwyn.org |- Configure WebSocket Proxy : True |- [ INFO ] Stage: Transaction setup |- [ INFO ] Stopping engine service |- [ INFO ] Stopping ovirt-fence-kdump-listener service |- [ INFO ] Stopping websocket-proxy service |- [ INFO ] Stage: Misc configuration |- [ INFO ] Stage: Package installation |- [ INFO ] Stage: Misc configuration |- [ INFO ] Initializing PostgreSQL |- [ INFO ] Creating PostgreSQL 'engine' database |- [ INFO ] Configuring PostgreSQL |- [ INFO ] Creating/refreshing Engine database schema |- [ INFO ] Creating/refreshing Engine 'internal' domain database schema |- [ INFO ] Upgrading CA |- [ INFO ] Creating CA |- [ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts |- [ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts |- [ INFO ] Configuring WebSocket Proxy |- [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' |- [ INFO ] Stage: Transaction commit |- [ INFO ] Stage: Closing up |- |- --== SUMMARY ==-- |- |- [WARNING] Less than 16384MB of memory is available |- SSH fingerprint: 13:dc:47:b9:38:66:15:eb:83:e8:ed:6f:71:cb:d4:ea |- Internal CA D8:4B:4C:74:45:EB:87:C4:8F:FE:DE:02:DF:99:5E:CB:AA:1B:D84 |- Note! If you want to gather statistical information you can install Reports and/or DWH: |- http://www.ovirt.org/Ovirt_DWH |- http://www.ovirt.org/Ovirt_Reports |- Web access is enabled at: |- http://ovirt.lutwyn.org:80/ovirt-engine |- https://ovirt.lutwyn.org:443/ovirt-engine |- Please use the user 'admin@internal' and password specified in order to login |- |- --== END OF SUMMARY ==-- |- |- [ INFO ] Starting engine service |- [ INFO ] Restarting httpd |- [ INFO ] Restarting ovirt-vmconsole proxy service |- [ INFO ] Stage: Clean up |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20160430234733-h0rns4.log |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20160430234907-setup.conf' |- [ INFO ] Stage: Pre-termination |- [ INFO ] Stage: Termination |- [ INFO ] Execution of setup completed successfully |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS [ INFO ] Engine-setup successfully completed [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Acquiring internal CA cert from the engine [ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct: [ INFO ] Issuer: C=US, O=lutwyn.org, CN=ovirt.lutwyn.org.36587, Subject: C=US, O=lutwyn.org, CN=ovirt.lutwyn.org.36587, Fingerprint (SHA-1): D84B4C7445EB87C48FFEDE02DF995ECBAA1BD84 [ INFO ] Connecting to the Engine [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] The VDSM Host is now operational [ INFO ] Saving hosted-engine configuration on the shared storage domain [ INFO ] Shutting down the engine VM [ INFO ] Enabling and starting HA services [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20160501015145.conf' [ INFO ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ INFO ] Hosted Engine successfully set up [root@ovirt01 ~]#
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Didi
On Mon, May 2, 2016 at 11:06 AM, Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, May 2, 2016 at 11:48 AM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, May 2, 2016 at 9:58 AM, Simone Tiraboschi wrote:
hosted-engine-setup creates a fresh VM and inject a cloud-init script to configure it and execute there engine-setup to configure the engine as needed. Since engine-setup is running on the engine VM triggered by cloud-init, hosted-engine-setup has no way to really control its process status so we simply gather its output with a timeout of 10 minutes between each single output line. In nothing happens within 10 minutes (the value is easily customizable), hosted-engine-setup thinks that engine-setup is stuck.
How can one customize the pre-set timeout?
To set 20 minutes you can pass this OVEHOSTED_ENGINE/engineSetupTimeout=int:1200
Could it be better to ask the user at the end of timeout if he/she wants to wait again, instead of directly fail?
Perhaps, can you please open a bz?
+1
So the issue we have to understood is why this simple command took more than 10 minutes in your env: 2016-04-30 17:56:57 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2216-03-13 17:56:57Z'), executable='None', cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH': '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**', 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/', 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE':
'/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log', 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly', 'OTOPI_EXECDIR': '/'}
It seemed quite strange to me too (see below further info on this)
Can you please check the entropy value on your host? cat /proc/sys/kernel/random/entropy_avail
I have not at hand now the server. I'll check soon and report Do you mean entropy of the physical server that will operate as hypervisor?
On the hypervisor
That's a good question. Simone - do you know if we start the guest with virtio-rng?
AFAIK we are not.
This is another case of [1], perhaps we should reopen it.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1319827
As a last question how to clean up things in case I have to start from scratch.
I'd recommend to redeploy from scratch instead of trying fixing it but, before that, we need to understand the root issue.
So, trying restart the setup with generated answer file I got: 1) if VM still powered on, an error about this condition 2) if VM powered down, an error abut storage domain already in place and restart not supported in this condition.
I was able to continue with these steps:
a) remove what inside the partially setup self hosted engine storage domain rm -rf /SHE_DOMAIN/* cd SHE_DOMAIN mklost+found
b) reboot the hypervisor
c) stop vdsmd
d) start the setup again with the answer file It seems all went well and this time strangely the step that took more than 10 minutes before lasted less than 2 seconds....
I was then able to deploy storage and iso domains without problems and self hosted engine domain correctly detected and imported too. Created two CentOS VMs without problems (6.7 and 7.2).
See below the full output of deploy command
[root@ovirt01 ~]# hosted-engine --deploy --config-append=/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf [ INFO ] Stage: Initializing [ INFO ] Generating a temporary VNC password. [ INFO ] Stage: Environment setup Configuration files: ['/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf'] Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20160501014326-8frbxk.log Version: otopi-1.4.1 (otopi-1.4.1-1.el7.centos) [ INFO ] Hardware supports virtualization [ INFO ] Bridge ovirtmgmt already created [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup [ INFO ] Stage: Environment customization
--== STORAGE CONFIGURATION ==--
During customization use CTRL-D to abort. [ INFO ] Installing on first host
--== SYSTEM CONFIGURATION ==--
--== NETWORK CONFIGURATION ==--
--== VM CONFIGURATION ==--
[ INFO ] Checking OVF archive content (could take a few minutes depending on archive size) [ INFO ] Checking OVF XML content (could take a few minutes depending on archive size) [WARNING] OVF does not contain a valid image description, using default. Enter root password that will be used for the engine appliance (leave it empty to skip): Confirm appliance root password: [ INFO ] The engine VM will be configured to use 192.168.1.212/24 The following CPU types are supported by this host: - model_Broadwell-noTSX: Intel Broadwell-noTSX Family - model_Haswell-noTSX: Intel Haswell-noTSX Family - model_SandyBridge: Intel SandyBridge Family - model_Westmere: Intel Westmere Family - model_Nehalem: Intel Nehalem Family - model_Penryn: Intel Penryn Family - model_Conroe: Intel Conroe Family [WARNING] Minimum requirements for CPUs not met [WARNING] Minimum requirements for disk size not met
--== HOSTED ENGINE CONFIGURATION ==--
Enter the name which will be used to identify this host inside the Administrator Portal [hosted_engine_1]: Enter 'admin@internal' user password that will be used for accessing the Administrator Portal: Confirm 'admin@internal' user password: [ INFO ] Stage: Setup validation
--== CONFIGURATION PREVIEW ==--
Bridge interface : eth0 Engine FQDN : ovirt.lutwyn.org Bridge name : ovirtmgmt Host address : ovirt01.lutwyn.org SSH daemon port : 22 Firewall manager : iptables Gateway address : 192.168.1.1 Host name for web application : hosted_engine_1 Host ID : 1 Image size GB : 10 GlusterFS Share Name : hosted_engine_glusterfs GlusterFS Brick Provisioning : False Storage connection : ovirt01.lutwyn.org:/SHE_DOMAIN Console type : vnc Memory size MB : 8192 MAC address : 00:16:3e:2e:93:15 Boot type : disk Number of CPUs : 1 OVF archive (for disk boot) : /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-3.6-20160420.1.el7.centos.ova Restart engine VM after engine-setup: True CPU Type : model_Broadwell-noTSX [ INFO ] Stage: Transaction setup [ INFO ] Stage: Misc configuration [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Configuring libvirt [ INFO ] Configuring VDSM [ INFO ] Starting vdsmd [ INFO ] Waiting for VDSM hardware info [ INFO ] Creating Storage Domain [ INFO ] Creating Storage Pool [ INFO ] Connecting Storage Pool [ INFO ] Verifying sanlock lockspace initialization [ INFO ] Creating Image for 'hosted-engine.lockspace' ... [ INFO ] Image for 'hosted-engine.lockspace' created successfully [ INFO ] Creating Image for 'hosted-engine.metadata' ... [ INFO ] Image for 'hosted-engine.metadata' created successfully [ INFO ] Creating VM Image [ INFO ] Extracting disk image from OVF archive (could take a few minutes depending on archive size) [ INFO ] Validating pre-allocated volume size [ INFO ] Uploading volume to data domain (could take a few minutes depending on archive size) [ INFO ] Image successfully imported from OVF [ INFO ] Destroying Storage Pool [ INFO ] Start monitoring domain [ INFO ] Configuring VM [ INFO ] Updating hosted-engine configuration [ INFO ] Stage: Transaction commit [ INFO ] Stage: Closing up [ INFO ] Creating VM You can now connect to the VM with the following command: /bin/remote-viewer vnc://localhost:5900 Use temporary password "4699ouyH" to connect to vnc console. Please note that in order to use remote-viewer you need to be able to run graphical applications. This means that if you are using ssh you have to supply the -Y flag (enables trusted X11 forwarding). Otherwise you can run the command from a terminal in your preferred desktop environment. If you cannot run graphical applications you can connect to the graphic console from another host or connect to the serial console using the following command: socat UNIX-CONNECT:/var/run/ovirt-vmconsole-console/4dae6016-ff01-4a1f-87ea-09f049d0e58c.sock,user=ovirt-vmconsole STDIO,raw,echo=0,escape=1 Please ensure that your Guest OS is properly configured to support serial console according to your distro documentation. Follow http://www.ovirt.org/Serial_Console_Setup#I_need_to_access_the_console_the_o... for more info. If you need to reboot the VM you will need to start it manually using the command: hosted-engine --vm-start You can then set a temporary password using the command: hosted-engine --add-console-password [ INFO ] Running engine-setup on the appliance |- [ INFO ] Stage: Initializing |- [ INFO ] Stage: Environment setup |- Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/root/ovirt-engine-answers', '/root/heanswers.conf'] |- Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20160430234733-h0rns4.log |- Version: otopi-1.4.1 (otopi-1.4.1-1.el7.centos) |- [ INFO ] Stage: Environment packages setup |- [ INFO ] Stage: Programs detection |- [ INFO ] Stage: Environment setup |- [ INFO ] Stage: Environment customization |- |- --== PRODUCT OPTIONS ==-- |- |- |- --== PACKAGES ==-- |- |- |- --== ALL IN ONE CONFIGURATION ==-- |- |- |- --== NETWORK CONFIGURATION ==-- |- |- [ INFO ] firewalld will be configured as firewall manager. |- |- --== DATABASE CONFIGURATION ==-- |- |- |- --== OVIRT ENGINE CONFIGURATION ==-- |- |- |- --== STORAGE CONFIGURATION ==-- |- |- |- --== PKI CONFIGURATION ==-- |- |- |- --== APACHE CONFIGURATION ==-- |- |- |- --== SYSTEM CONFIGURATION ==-- |- |- |- --== MISC CONFIGURATION ==-- |- |- |- --== END OF CONFIGURATION ==-- |- |- [ INFO ] Stage: Setup validation |- [WARNING] Cannot validate host name settings, reason: resolved host does not match any of the local addresses |- [WARNING] Less than 16384MB of memory is available |- |- --== CONFIGURATION PREVIEW ==-- |- |- Application mode : virt |- Default SAN wipe after delete : False |- Firewall manager : firewalld |- Update Firewall : True |- Host FQDN : ovirt.lutwyn.org |- Engine database secured connection : False |- Engine database host : localhost |- Engine database user name : engine |- Engine database name : engine |- Engine database port : 5432 |- Engine database host name validation : False |- Engine installation : True |- PKI organization : lutwyn.org |- Configure local Engine database : True |- Set application as default page : True |- Configure Apache SSL : True |- Configure VMConsole Proxy : True |- Engine Host FQDN : ovirt.lutwyn.org |- Configure WebSocket Proxy : True |- [ INFO ] Stage: Transaction setup |- [ INFO ] Stopping engine service |- [ INFO ] Stopping ovirt-fence-kdump-listener service |- [ INFO ] Stopping websocket-proxy service |- [ INFO ] Stage: Misc configuration |- [ INFO ] Stage: Package installation |- [ INFO ] Stage: Misc configuration |- [ INFO ] Initializing PostgreSQL |- [ INFO ] Creating PostgreSQL 'engine' database |- [ INFO ] Configuring PostgreSQL |- [ INFO ] Creating/refreshing Engine database schema |- [ INFO ] Creating/refreshing Engine 'internal' domain database schema |- [ INFO ] Upgrading CA |- [ INFO ] Creating CA |- [ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts |- [ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts |- [ INFO ] Configuring WebSocket Proxy |- [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' |- [ INFO ] Stage: Transaction commit |- [ INFO ] Stage: Closing up |- |- --== SUMMARY ==-- |- |- [WARNING] Less than 16384MB of memory is available |- SSH fingerprint: 13:dc:47:b9:38:66:15:eb:83:e8:ed:6f:71:cb:d4:ea |- Internal CA D8:4B:4C:74:45:EB:87:C4:8F:FE:DE:02:DF:99:5E:CB:AA:1B:D84 |- Note! If you want to gather statistical information you can install Reports and/or DWH: |- http://www.ovirt.org/Ovirt_DWH |- http://www.ovirt.org/Ovirt_Reports |- Web access is enabled at: |- http://ovirt.lutwyn.org:80/ovirt-engine |- https://ovirt.lutwyn.org:443/ovirt-engine |- Please use the user 'admin@internal' and password specified in order to login |- |- --== END OF SUMMARY ==-- |- |- [ INFO ] Starting engine service |- [ INFO ] Restarting httpd |- [ INFO ] Restarting ovirt-vmconsole proxy service |- [ INFO ] Stage: Clean up |- Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20160430234733-h0rns4.log |- [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20160430234907-setup.conf' |- [ INFO ] Stage: Pre-termination |- [ INFO ] Stage: Termination |- [ INFO ] Execution of setup completed successfully |- HE_APPLIANCE_ENGINE_SETUP_SUCCESS [ INFO ] Engine-setup successfully completed [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine is still unreachable [ INFO ] Engine is still not reachable, waiting... [ INFO ] Engine replied: DB Up!Welcome to Health Status! [ INFO ] Acquiring internal CA cert from the engine [ INFO ] The following CA certificate is going to be used, please immediately interrupt if not correct: [ INFO ] Issuer: C=US, O=lutwyn.org, CN=ovirt.lutwyn.org.36587, Subject: C=US, O=lutwyn.org, CN=ovirt.lutwyn.org.36587, Fingerprint (SHA-1): D84B4C7445EB87C48FFEDE02DF995ECBAA1BD84 [ INFO ] Connecting to the Engine [ INFO ] Waiting for the host to become operational in the engine. This may take several minutes... [ INFO ] The VDSM Host is now operational [ INFO ] Saving hosted-engine configuration on the shared storage domain [ INFO ] Shutting down the engine VM [ INFO ] Enabling and starting HA services [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20160501015145.conf' [ INFO ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ INFO ] Hosted Engine successfully set up [root@ovirt01 ~]#
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Didi
On Mon, May 2, 2016 at 11:14 AM, Simone Tiraboschi <stirabos@redhat.com> wrote:
Can you please check the entropy value on your host? cat /proc/sys/kernel/random/entropy_avail
I have not at hand now the server. I'll check soon and report Do you mean entropy of the physical server that will operate as hypervisor?
On the hypervisor
That's a good question. Simone - do you know if we start the guest with virtio-rng?
AFAIK we are not.
On the only existing hypervisor, just after booting and exiting global maintenance, causing hosted engine to start, I have [root@ovirt01 ~]# uptime 20:34:17 up 6 min, 1 user, load average: 0.23, 0.20, 0.11 [root@ovirt01 ~]# cat /proc/sys/kernel/random/entropy_avail 3084 BTW on the self hosted engine VM: [root@ovirt ~]# uptime 18:35:33 up 4 min, 1 user, load average: 0.06, 0.25, 0.13 [root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail 14 On the hypervisor: [root@ovirt01 ~]# ps -ef | grep [q]emu | grep virtio-rng [root@ovirt01 ~]# On engine VM: [root@ovirt ~]# ll /dev/hwrng ls: cannot access /dev/hwrng: No such file or directory [root@ovirt ~]# [root@ovirt ~]# lsmod | grep virtio_rng [root@ovirt ~]# May I change anything so that engine VM has virtio-rng enabled? Gianluca
On Mon, May 2, 2016 at 8:39 PM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, May 2, 2016 at 11:14 AM, Simone Tiraboschi <stirabos@redhat.com> wrote:
Can you please check the entropy value on your host? cat /proc/sys/kernel/random/entropy_avail
I have not at hand now the server. I'll check soon and report Do you mean entropy of the physical server that will operate as hypervisor?
On the hypervisor
That's a good question. Simone - do you know if we start the guest with virtio-rng?
AFAIK we are not.
On the only existing hypervisor, just after booting and exiting global maintenance, causing hosted engine to start, I have
[root@ovirt01 ~]# uptime 20:34:17 up 6 min, 1 user, load average: 0.23, 0.20, 0.11
[root@ovirt01 ~]# cat /proc/sys/kernel/random/entropy_avail 3084
BTW on the self hosted engine VM: [root@ovirt ~]# uptime 18:35:33 up 4 min, 1 user, load average: 0.06, 0.25, 0.13
[root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail 14
On the hypervisor: [root@ovirt01 ~]# ps -ef | grep [q]emu | grep virtio-rng [root@ovirt01 ~]#
On engine VM: [root@ovirt ~]# ll /dev/hwrng ls: cannot access /dev/hwrng: No such file or directory [root@ovirt ~]#
[root@ovirt ~]# lsmod | grep virtio_rng [root@ovirt ~]#
May I change anything so that engine VM has virtio-rng enabled?
Gianluca
I verified very slow login time in webadmin after welcome page, with my configuration that is for now based on /etc/hosts. After reading a previous post, and having after about 30 minutes only 114 as entropy in hosted engine vm, I made this in engine VM: yum install haveged systemctl enable haveged put host in global maintenance shutdown engine VM exit from maintenance engine VM starts and immediately I have: [root@ovirt ~]# uptime 19:05:10 up 0 min, 1 user, load average: 0.68, 0.20, 0.07 [root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail 1369 And login in web admin page now almost immediate.... Inside the thread I read: http://lists.ovirt.org/pipermail/users/2016-April/038805.html it wasn't clear if I can edit the engine VM in webadmin (or other mean) and enable the random generator option or if the haveged way is the one to go with in case of self hosted engine.... Is there a list of what I can change (if any) and what not for the engine VM? For example I would like to change the time zone that is GMT now (I think inherited from the OVF of the appliance?) Thanks, Gianluca
On Mon, May 2, 2016 at 9:15 PM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, May 2, 2016 at 8:39 PM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, May 2, 2016 at 11:14 AM, Simone Tiraboschi <stirabos@redhat.com> wrote:
Can you please check the entropy value on your host? cat /proc/sys/kernel/random/entropy_avail
I have not at hand now the server. I'll check soon and report Do you mean entropy of the physical server that will operate as hypervisor?
On the hypervisor
That's a good question. Simone - do you know if we start the guest with virtio-rng?
AFAIK we are not.
On the only existing hypervisor, just after booting and exiting global maintenance, causing hosted engine to start, I have
[root@ovirt01 ~]# uptime 20:34:17 up 6 min, 1 user, load average: 0.23, 0.20, 0.11
[root@ovirt01 ~]# cat /proc/sys/kernel/random/entropy_avail 3084
BTW on the self hosted engine VM: [root@ovirt ~]# uptime 18:35:33 up 4 min, 1 user, load average: 0.06, 0.25, 0.13
[root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail 14
On the hypervisor: [root@ovirt01 ~]# ps -ef | grep [q]emu | grep virtio-rng [root@ovirt01 ~]#
On engine VM: [root@ovirt ~]# ll /dev/hwrng ls: cannot access /dev/hwrng: No such file or directory [root@ovirt ~]#
[root@ovirt ~]# lsmod | grep virtio_rng [root@ovirt ~]#
May I change anything so that engine VM has virtio-rng enabled?
Gianluca
I verified very slow login time in webadmin after welcome page, with my configuration that is for now based on /etc/hosts. After reading a previous post, and having after about 30 minutes only 114 as entropy in hosted engine vm, I made this in engine VM:
Thanks for your report Gianluca, adding virtio-rng or adding haveged daemon to the appliance is indeed a good idea: could you please fill an RFE on bugzilla for that?
yum install haveged systemctl enable haveged
put host in global maintenance shutdown engine VM exit from maintenance
engine VM starts and immediately I have:
[root@ovirt ~]# uptime 19:05:10 up 0 min, 1 user, load average: 0.68, 0.20, 0.07
[root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail 1369
And login in web admin page now almost immediate....
Inside the thread I read: http://lists.ovirt.org/pipermail/users/2016-April/038805.html
it wasn't clear if I can edit the engine VM in webadmin (or other mean) and enable the random generator option or if the haveged way is the one to go with in case of self hosted engine.... Is there a list of what I can change (if any) and what not for the engine VM? For example I would like to change the time zone that is GMT now (I think inherited from the OVF of the appliance?)
Thanks, Gianluca
participants (3)
-
Gianluca Cecchi -
Simone Tiraboschi -
Yedidyah Bar David