Cannot retrieve answer file from 1st HE host when setting up 2nd host
Doing a “hosted-engine —setup” on my 2nd oVirt host, and am getting the following error: --== SYSTEM CONFIGURATION ==-- [WARNING] A configuration file must be supplied to deploy Hosted Engine on an additional host. The answer file may be fetched from the first host using scp. If you do not want to download it automatically you can abort the setup answering no to the following question. Do you want to scp the answer file from the first host? (Yes, No)[Yes]: Please provide the FQDN or IP of the first host: 249.26.200.91 Enter 'root' user password for host 249.26.200.91: [ ERROR ] Failed to execute stage 'Environment customization': [Errno 2] No such file [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20151218205656.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination The problem may be slowness of the SSH connection to the 1st host; ever since I did the hosted engine setup on the 1st host and it created the ‘ovirtmgmt’ bridge et al, it now for some reason takes a long time (~50 sec’s) to SSH into the host… Example: (key exchange was first done from 2nd host to first host) [root@ovirt-node-02 ~]# time ssh root@ovirt-node-01 uptime 21:37:33 up 1 day, 10:43, 0 users, load average: 0.14, 0.15, 0.14 real 0m50.185s user 0m0.048s sys 0m0.006s So maybe because of that reason, the setup SCP session is timing out?? How can I fix this?
On 19-12-2015 3:42, Will Dennis wrote:
The problem may be slowness of the SSH connection to the 1st host; ever since I did the hosted engine setup on the 1st host and it created the ‘ovirtmgmt’ bridge et al, it now for some reason takes a long time (~50 sec’s) to SSH into the host… Example: (key exchange was first done from 2nd host to first host)
[root@ovirt-node-02 ~]# time ssh root@ovirt-node-01 uptime 21:37:33 up 1 day, 10:43, 0 users, load average: 0.14, 0.15, 0.14
real 0m50.185s user 0m0.048s sys 0m0.006s
So maybe because of that reason, the setup SCP session is timing out?? How can I fix this? _______________________________________________
Are you sure you're dns is setup correctly. It looks like that node01 is looking up or reverse looking up node02 and that can take a long time if dns isn't working properly. Regards, Joop
Hi Joop, I thought about that, but DNS resolution seems to be fine… [root@ovirt-node-01 ~]# time dig ovirt-node-02.nec-labs.com ; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7_2.1 <<>> ovirt-node-02.nec-labs.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4574 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;ovirt-node-02.nec-labs.com. IN A ;; ANSWER SECTION: ovirt-node-02.nec-labs.com. 3600 IN A 249.26.200.92 ;; Query time: 0 msec ;; SERVER: 249.26.108.11#53(249.26.108.11) ;; WHEN: Sat Dec 19 21:30:42 EST 2015 ;; MSG SIZE rcvd: 71 real 0m0.010s user 0m0.005s sys 0m0.005s [root@ovirt-node-01 ~]# time dig -x 249.26.200.92 ; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7_2.1 <<>> -x 249.26.200.92 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65149 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;92.200.15.138.in-addr.arpa. IN PTR ;; ANSWER SECTION: 92.200.15.138.in-addr.arpa. 3600 IN PTR ovirt-node-02.nec-labs.com. ;; Query time: 0 msec ;; SERVER: 249.26.108.11#53(249.26.108.11) ;; WHEN: Sat Dec 19 21:31:18 EST 2015 ;; MSG SIZE rcvd: 95 real 0m0.010s user 0m0.007s sys 0m0.002s
OK, after all the t’shooting I could think to do, out of desperation I put the engine VM in maint mode, then rebooted the host. When it came up, I can now SSH into it normally (no long delay now.) I saw two state email changes come in after the reboot: - StartState-ReinitializeFSM - ReinitializeFSM-LocalMaintenance But the engine VM never started… Do I have to do something at the host (issue a command) to get the engine VM to start? (Sorry if I’m ignorant, an oVirt noob here…) -Will On Dec 19, 2015, at 9:38 PM, Will Dennis <wdennis@nec-labs.com<mailto:wdennis@nec-labs.com>> wrote: Hi Joop, I thought about that, but DNS resolution seems to be fine… [root@ovirt-node-01 ~]# time dig ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com> ; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7_2.1 <<>> ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com> ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4574 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com>. IN A ;; ANSWER SECTION: ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com>. 3600 IN A 249.26.200.92 ;; Query time: 0 msec ;; SERVER: 249.26.108.11#53(249.26.108.11) ;; WHEN: Sat Dec 19 21:30:42 EST 2015 ;; MSG SIZE rcvd: 71 real 0m0.010s user 0m0.005s sys 0m0.005s [root@ovirt-node-01 ~]# time dig -x 249.26.200.92 ; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7_2.1 <<>> -x 249.26.200.92 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65149 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;92.200.15.138.in-addr.arpa. IN PTR ;; ANSWER SECTION: 92.200.15.138.in-addr.arpa. 3600 IN PTR ovirt-node-02.nec-labs.com<http://ovirt-node-02.nec-labs.com>. ;; Query time: 0 msec ;; SERVER: 249.26.108.11#53(249.26.108.11) ;; WHEN: Sat Dec 19 21:31:18 EST 2015 ;; MSG SIZE rcvd: 95 real 0m0.010s user 0m0.007s sys 0m0.002s
On December 20, 2015 5:19:43 AM CET, Will Dennis <wdennis@nec-labs.com> wrote:
OK, after all the t’shooting I could think to do, out of desperation I put the engine VM in maint mode, then rebooted the host. When it came up, I can now SSH into it normally (no long delay now.) I saw two state email changes come in after the reboot:
- StartState-ReinitializeFSM - ReinitializeFSM-LocalMaintenance
But the engine VM never started… Do I have to do something at the host (issue a command) to get the engine VM to start? (Sorry if I’m ignorant, an oVirt noob here…) No problem, we were all once.
Run hosted-engine --set-maintenance -mode=none After a couple of minutes you should get mail about engine events. Just curious. What specs sre your host (s) and engine vm? Regards, Joop
I have three hosts, each with the following specs: CPU: (2) Intel Xeon X5450 @ 3.00GHz (8 cores total) RAM: 24GB Disk: 2 Gluster volumes (each distributed w/ 3 replicas) for oVirt; engine: 1.9TB, vmdata: 3.7TB I have allocated 8192MB RAM to the hosted engine. -Will
On Dec 20, 2015, at 3:17 AM, Joop van de Wege <jvdwege@xs4all.nl> wrote: No problem, we were all once.
Just curious. What specs sre your host (s) and engine vm?
Regards,
Joop
OK, thanks to the assist from Joob, I did startup the engine VM, and took it out of maint mode… Then I went to my second oVirt host, and tried “hosted-engine —setup” again, and I’m still getting the error message “Failed to execute stage 'Environment customization': [Errno 2] No such file” Screen output of the “hosted-engine —setup” run: http://ur1.ca/ocr5u So, I guess that means it can’t find the answer file where it expects it on the first host… What do I do now? -Will On Dec 19, 2015, at 11:20 PM, Will Dennis <wdennis@nec-labs.com<mailto:wdennis@nec-labs.com>> wrote: OK, after all the t’shooting I could think to do, out of desperation I put the engine VM in maint mode, then rebooted the host. When it came up, I can now SSH into it normally (no long delay now.) I saw two state email changes come in after the reboot: - StartState-ReinitializeFSM - ReinitializeFSM-LocalMaintenance But the engine VM never started… Do I have to do something at the host (issue a command) to get the engine VM to start? (Sorry if I’m ignorant, an oVirt noob here…) -Will
FYI, the answer file is located at: /var/lib/ovirt-hosted-engine-setup/answers/answers-20151218124259.conf There are many other answer files in that directory as well from prior unsuccessful runs… Does that matter? On Dec 20, 2015, at 2:51 PM, Will Dennis <wdennis@nec-labs.com<mailto:wdennis@nec-labs.com>> wrote: OK, thanks to the assist from Joob, I did startup the engine VM, and took it out of maint mode… Then I went to my second oVirt host, and tried “hosted-engine —setup” again, and I’m still getting the error message “Failed to execute stage 'Environment customization': [Errno 2] No such file” Screen output of the “hosted-engine —setup” run: http://ur1.ca/ocr5u So, I guess that means it can’t find the answer file where it expects it on the first host… What do I do now? -Will
On Sun, Dec 20, 2015 at 10:04 PM, Will Dennis <wdennis@nec-labs.com> wrote:
FYI, the answer file is located at: /var/lib/ovirt-hosted-engine-setup/answers/answers-20151218124259.conf
There are many other answer files in that directory as well from prior unsuccessful runs… Does that matter?
The answerfile is expected to be found in /etc/ovirt-hosted-engine/answers.conf . It's written there upon a successful completion of deploy. It's also always written to /var/lib/ovirt-hosted-engine-setup/answers/ . You can also copy it by yourself to somefile and run: hosted-engine --deploy --config-append=somefile But I'd invest some time understanding why scp fails. Might be bad name resolution?
On Dec 20, 2015, at 2:51 PM, Will Dennis <wdennis@nec-labs.com<mailto:wdennis@nec-labs.com>> wrote:
OK, thanks to the assist from Joob, I did startup the engine VM, and took it out of maint mode… Then I went to my second oVirt host, and tried “hosted-engine —setup” again, and I’m still getting the error message “Failed to execute stage 'Environment customization': [Errno 2] No such file”
Screen output of the “hosted-engine —setup” run: http://ur1.ca/ocr5u
So, I guess that means it can’t find the answer file where it expects it on the first host… What do I do now?
-Will
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Didi
On Mon, Dec 21, 2015 at 9:33 AM, Yedidyah Bar David <didi@redhat.com> wrote:
On Sun, Dec 20, 2015 at 10:04 PM, Will Dennis <wdennis@nec-labs.com> wrote:
FYI, the answer file is located at: /var/lib/ovirt-hosted-engine-setup/answers/answers-20151218124259.conf
There are many other answer files in that directory as well from prior unsuccessful runs… Does that matter?
The answerfile is expected to be found in /etc/ovirt-hosted-engine/answers.conf . It's written there upon a successful completion of deploy.
Absolutely, I suggest you to start checking for /etc/ovirt-hosted-engine/answers.conf on the first host. If the file is not here please check the setup logs on the first to ensure that it successful completed.
It's also always written to /var/lib/ovirt-hosted-engine-setup/answers/ .
You can also copy it by yourself to somefile and run:
hosted-engine --deploy --config-append=somefile
But I'd invest some time understanding why scp fails. Might be bad name resolution?
On Dec 20, 2015, at 2:51 PM, Will Dennis <wdennis@nec-labs.com<mailto:
wdennis@nec-labs.com>> wrote:
OK, thanks to the assist from Joob, I did startup the engine VM, and
took it out of maint mode… Then I went to my second oVirt host, and tried “hosted-engine —setup” again, and I’m still getting the error message “Failed to execute stage 'Environment customization': [Errno 2] No such file”
Screen output of the “hosted-engine —setup” run: http://ur1.ca/ocr5u
So, I guess that means it can’t find the answer file where it expects it
on the first host… What do I do now?
-Will
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Didi _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On December 20, 2015 8:50:06 PM CET, Will Dennis <wdennis@nec-labs.com> wrote:
OK, thanks to the assist from Joob, I did startup the engine VM, and took it out of maint mode… Then I went to my second oVirt host, and tried “hosted-engine —setup” again, and I’m still getting the error message “Failed to execute stage 'Environment customization': [Errno 2] No such file”
Screen output of the “hosted-engine —setup” run: http://ur1.ca/ocr5u
Can you provide the corresponding log file? I suspect that the scp failed. Do you allow root login with password on your nodes. Regards, Joop
participants (5)
-
Joop -
Joop van de Wege
-
Simone Tiraboschi -
Will Dennis -
Yedidyah Bar David