3:40 p.m.
--------------080102010006090400050504
Content-Type: text/plain; charset="ISO-8859-1"; format=flowed
Content-Transfer-Encoding: 7bit
Current setup:
* 3 identical hosts running on HP GL180 g5 servers
o gluster running 5 volumes in replica 3
* engine running on VMWare Server on another computer (that computer is NOT
available to convert to a host)
Where I want to end up:
* 3 identical hosted-engine hosts running on HP GL180 g5 servers
o gluster running 6 volumes in replica 3
+ new volume will be nfs storage for engine VM
* hosted engine in oVirt VM
* as few changes to current setup as possible
The two pages I found on the wiki are: Hosted Engine Howto
<http://www.ovirt.org/Hosted_Engine_Howto> and Migrate to Hosted Engine
<http://www.ovirt.org/Migrate_to_Hosted_Engine>;. Both were written during
the testing process, and have not been updated to reflect production status.
I don't know if anything in the process has changed since they were written.
Process outlined in above two pages (as I understand it):
have nfs file store ready to hold VM
Do minimal install (not clear if ovirt node, Centos, or Fedora was
used--I am Centos-based)
# yum install ovirt-hosted-engine-setup
# hosted-engine --deploy
Install OS on VM
return to host console
at "Please install the engine in the VM" prompt on host
on VM console
# yum install ovirt-engine
on old engine:
service ovirt-engine stop
chkconfig ovirt-engine off
set up dns for new engine
# engine-backup --mode=backup --file=backup1 --log=backup1.log
scp backup file to new engine VM
on new VM:
# engine-backup --mode=restore --file=backup1 --log=backup1-restore.log
--change-db-credentials --db-host=didi-lap --db-user=engine --db-password
--db-name=engine
# engine-setup
on host:
run script until: "The system will wait until the VM is down."
on new VM:
# reboot
on Host: finish script
My questions:
1. Is the above still the recommended way to do a hosted-engine install?
2. Will it blow up at me if I use my existing host (with glusterfs all set
up, etc) as the starting point, instead of a clean install?
Thank you for letting me benefit from your experience,
Ted Miller
Elkhart, IN, USA
--------------080102010006090400050504
Content-Type: text/html; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Current setup:<br>
<ul>
<li>3 identical hosts running on HP GL180 g5 servers</li>
<ul>
<li>gluster running 5 volumes in replica 3<br>
</li>
</ul>
<li>engine running on VMWare Server on another computer (that
computer is NOT available to convert to a host)<br>
</li>
</ul>
Where I want to end up:<br>
<ul>
<li>3 identical hosted-engine hosts running on HP GL180 g5 servers</li>
<ul>
<li>gluster running 6 volumes in replica 3</li>
<ul>
<li>new volume will be nfs storage for engine VM</li>
</ul>
</ul>
<li>hosted engine in oVirt VM</li>
<li>as few changes to current setup as possible</li>
</ul>
<p>The two pages I found on the wiki are: <a
href="http://www.ovirt.org/Hosted_Engine_Howto">Hosted Engine
Howto</a> and <a
href="http://www.ovirt.org/Migrate_to_Hosted_Engine">Migrate to
Hosted Engine</a>. Both were written during the testing
process, and have not been updated to reflect production status.
I don't know if anything in the process has changed since they
were written.<br>
</p>
<p>Process outlined in above two pages (as I understand it):<br>
</p>
<blockquote>
<p>have nfs file store ready to hold VM</p>
<p>Do minimal install (not clear if ovirt node, Centos, or Fedora
was used--I am Centos-based)</p>
# yum install ovirt-hosted-engine-setup<br>
# hosted-engine --deploy<br>
<p>Install OS on VM<br>
</p>
<p>return to host console<br>
</p>
<p>at "Please install the engine in the VM" prompt on
host<br>
</p>
<p>on VM console<br>
# yum install ovirt-engine<br>
</p>
<p>on old engine: <br>
service ovirt-engine stop<br>
chkconfig ovirt-engine off</p>
<p>set up dns for new engine<br>
</p>
<p># engine-backup --mode=backup --file=backup1 --log=backup1.log<br>
scp backup file to new engine VM<br>
</p>
<p>on new VM:<br>
# engine-backup --mode=restore --file=backup1
--log=backup1-restore.log --change-db-credentials
--db-host=didi-lap --db-user=engine --db-password
--db-name=engine<br>
# engine-setup<br>
</p>
<p>on host:<br>
run script until: "The system will wait until the VM is
down."<br>
</p>
<p>on new VM:<br>
# reboot<br>
</p>
<p>on Host: finish script<br>
</p>
</blockquote>
My questions:<br>
<br>
1. Is the above still the recommended way to do a hosted-engine
install?<br>
<br>
2. Will it blow up at me if I use my existing host (with glusterfs
all set up, etc) as the starting point, instead of a clean install?<br>
<br>
Thank you for letting me benefit from your experience,<br>
Ted Miller<br>
Elkhart, IN, USA<br>
<br>
</body>
</html>
--------------080102010006090400050504--
1:58 a.m.
----- Original Message -----
From: "Ted Miller" <tmiller(a)hcjb.org>
To: "users" <users(a)ovirt.org>
Sent: Tuesday, April 1, 2014 10:40:38 PM
Subject: [Users] Migrate cluster 3.3 -> 3.4 hosted on existing hosts
Current setup:
* 3 identical hosts running on HP GL180 g5 servers
* gluster running 5 volumes in replica 3
* engine running on VMWare Server on another computer (that computer is
NOT available to convert to a host)
Where I want to end up:
* 3 identical hosted-engine hosts running on HP GL180 g5 servers
* gluster running 6 volumes in replica 3
* new volume will be nfs storage for engine VM
* hosted engine in oVirt VM * as few changes to current setup as
possible
The two pages I found on the wiki are: Hosted Engine Howto and Migrate to
Hosted Engine . Both were written during the testing process, and have not
been updated to reflect production status. I don't know if anything in the
process has changed since they were written.
Basically things remained the same, with some details changing perhaps.
Process outlined in above two pages (as I understand it):
have nfs file store ready to hold VM
Do minimal install (not clear if ovirt node, Centos, or Fedora was used--I am
Centos-based)
Fedora/Centos/RHEL are supposed to work. ovirt node is currently not
supported - iirc it's planned to be supported soon, not sure.
# yum install ovirt-hosted-engine-setup
# hosted-engine --deploy
Install OS on VM
return to host console
at "Please install the engine in the VM" prompt on host
on VM console
# yum install ovirt-engine
on old engine:
service ovirt-engine stop
chkconfig ovirt-engine off
set up dns for new engine
# engine-backup --mode=backup --file=backup1 --log=backup1.log
scp backup file to new engine VM
on new VM:
Please see [1]. Specifically, if you had a local db, you'll first have
to create it yourself.
[1] http://www.ovirt.org/Ovirt-engine-backup#Howto
# engine-backup --mode=restore --file=backup1
--log=backup1-restore.log
--change-db-credentials --db-host=didi-lap --db-user=engine --db-password
--db-name=engine
The above assumes a db was already created and ready to use (access etc)
using the supplied credentials. You'll naturally have to provide your own.
# engine-setup
on host:
run script until: "The system will wait until the VM is down."
on new VM:
# reboot
on Host: finish script
My questions:
1. Is the above still the recommended way to do a hosted-engine install?
Yes.
2. Will it blow up at me if I use my existing host (with glusterfs all set
up, etc) as the starting point, instead of a clean install?
a. Probably yes, for now. I did not hear much about testing such a migration
using an existing host - ovirt or gluster or both. I did not test that myself
either.
If at all possible, you should use a new clean host. Do plan well and test.
Also see discussions on the mailing lists, e.g. this one:
http://lists.ovirt.org/pipermail/users/2014-March/thread.html#22441
Good luck, and please report back!
--
Didi
2:40 p.m.
On 4/2/2014 1:58 AM, Yedidyah Bar David wrote:
----- Original Message -----
> From: "Ted Miller" <tmiller(a)hcjb.org>
> To: "users" <users(a)ovirt.org>
> Sent: Tuesday, April 1, 2014 10:40:38 PM
> Subject: [Users] Migrate cluster 3.3 -> 3.4 hosted on existing hosts
>
> Current setup:
> * 3 identical hosts running on HP GL180 g5 servers
> * gluster running 5 volumes in replica 3
> * engine running on VMWare Server on another computer (that computer is
> NOT available to convert to a host)
>
> Where I want to end up:
> * 3 identical hosted-engine hosts running on HP GL180 g5 servers
> * gluster running 6 volumes in replica 3
> * new volume will be nfs storage for engine VM
> * hosted engine in oVirt VM
> * as few changes to current setup as possible
>
> The two pages I found on the wiki are: Hosted Engine Howto and Migrate to
> Hosted Engine . Both were written during the testing process, and have not
> been updated to reflect production status. I don't know if anything in the
> process has changed since they were written.
Basically things remained the same, with some details changing perhaps.
> Process outlined in above two pages (as I understand it):
>
> have nfs file store ready to hold VM
>
> Do minimal install (not clear if ovirt node, Centos, or Fedora was used--I am
> Centos-based)
Fedora/Centos/RHEL are supposed to work. ovirt node is currently not
supported - iirc it's planned to be supported soon, not sure.
> # yum install ovirt-hosted-engine-setup
> # hosted-engine --deploy
>
>
> Install OS on VM
>
>
> return to host console
>
>
> at "Please install the engine in the VM" prompt on host
>
>
> on VM console
> # yum install ovirt-engine
>
>
> on old engine:
> service ovirt-engine stop
> chkconfig ovirt-engine off
>
> set up dns for new engine
>
>
> # engine-backup --mode=backup --file=backup1 --log=backup1.log
> scp backup file to new engine VM
>
>
> on new VM:
Please see [1]. Specifically, if you had a local db, you'll first have
to create it yourself.
[1] http://www.ovirt.org/Ovirt-engine-backup#Howto
> # engine-backup --mode=restore --file=backup1 --log=backup1-restore.log
> --change-db-credentials --db-host=didi-lap --db-user=engine --db-password
> --db-name=engine
The above assumes a db was already created and ready to use (access etc)
using the supplied credentials. You'll naturally have to provide your own.
> # engine-setup
>
> on host:
> run script until: "The system will wait until the VM is down."
>
> on new VM:
> # reboot
>
> on Host: finish script
> My questions:
>
> 1. Is the above still the recommended way to do a hosted-engine install?
Yes.
> 2. Will it blow up at me if I use my existing host (with glusterfs all set
> up, etc) as the starting point, instead of a clean install?
a. Probably yes, for now. I did not hear much about testing such a migration
using an existing host - ovirt or gluster or both. I did not test that myself
either.
If at all possible, you should use a new clean host. Do plan well and test.
Also see discussions on the mailing lists, e.g. this one:
http://lists.ovirt.org/pipermail/users/2014-March/thread.html#22441
Good luck, and please report back!
I have good news and bad news.
I migrated the 3 host cluster from 3.4 to 3.4 hosted. The process went
fairly smoothly. Engine ran, I was able to add the three hosts to the
engine's domain, etc. That was all working about Thursday. (I did not get
fencing set up).
Friday, at the end of the day, I shut down the entire system (it is not yet
in production) because I was leaving for a week's vacation/holiday. I am
fairly certain that I put the system into global maintenance mode before
shutting down. I know I shut down the engine before shutting down the hosts.
Monday (10 days later) I came back from vacation and powered up the three
machines. The hosts came up fine, but the engine will not start. (I found
some gluster split-brain errors, and chased that for a couple of days, until
I realized that the split-brain was not the fundamental problem.)
During bootup /var/log/messages shows:
May 21 19:22:00 s2 ovirt-ha-broker mgmt_bridge.MgmtBridge ERROR Failed to
getVdsCapabilities: VDSM initialization timeout
May 21 19:22:00 s2 ovirt-ha-broker mem_free.MemFree ERROR Failed to getVdsStats: VDSM
initialization timeout
May 21 19:22:00 s2 ovirt-ha-broker cpu_load_no_engine.EngineHealth ERROR Failed to
getVmStats: VDSM initialization timeout
May 21 19:22:00 s2 ovirt-ha-broker engine_health.CpuLoadNoEngine ERROR Failed to
getVmStats: VDSM initialization timeout
May 21 19:22:03 s2 vdsm vds WARNING Unable to load the json rpc server module. Please make
sure it is installed.
and then /var/log/ovirt-hosted-engine-ha/agent.log shows:
MainThread::ERROR::2014-05-21
19:22:04,198::hosted_engine::414::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
Failed trying to connect storage:
MainThread::CRITICAL::2014-05-21
19:22:04,199::agent::103::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Could not start
ha-agent
Traceback (most recent call last):
File
"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line
97, in run self._run_agent()
File
"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line
154, in _run_agent hosted_engine.HostedEngine(self.shutdown_requested).start_monitoring()
File
"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 299, in start_monitoring self._initialize_vdsm()
File
"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 415, in _initialize_vdsm
raise Exception("Failed trying to connect storage")
Exception: Failed trying to connect storage
I can manually mount and start storage, but then my
sanlock/gluster/split-brain bites me, and there are no instructions out there
on how to recover from this situation.
I am starting over the "sanitary" way. The only thing getting reused is the
gluster file system. I will migrate one host at a time, and will get the
gluster system running under the hosts as I go, adding the host back into the
system as I go along.
You were right, :(
Ted Miller
Elkhart, IN, USA
P.S. I hope that the install scripts can eventually be modified to migrate
existing hosts, but I understand that first you must have it stable from a
clean install.
3872
days inactive
3924
days old
2 comments
2 participants
participants (2)
-
Ted Miller -
Yedidyah Bar David