7:57 p.m.
Hi Guys,
I encountered an unfortunate circumstance today. Possibly an achillies
heel.
I have three hypervisors, HV1, HV2, HV3, all running gluster for hosted
engine support. Individually they all pointed to HV1:/hosted_engine with
backupvol=HV2,HV3...
HV1 lost its bootsector, which was discovered upon a reboot. This had
zero impact, as designed, on the VM's.
However, now that HV1 is down, how does one go about replacing the
original HV? The backup servers point to HV1, and you cannot readd the
HV through the GUI, and the CLI will not readd it as it's already
there... you cannot remove it as it is down in the GUI...
Pointing the other HV's to their own storage may make sense for multiple
instances of the hosted_engine, however it's nice that the gluster
volumes are replicated and that one VM can be relaunched when a HV error
is detected. It's also consuming less resources.
What's the procedure to replace the original VM?
12:57 a.m.
Hi,
Pad [1] contains the procedure to replace the host with same FQDN
where existing host OS has to be re-installed.
[1] https://paste.fedoraproject.org/431252/47435076/
Thanks
kasturi.
On 09/20/2016 06:27 AM, Hanson wrote:
Hi Guys,
I encountered an unfortunate circumstance today. Possibly an achillies
heel.
I have three hypervisors, HV1, HV2, HV3, all running gluster for
hosted engine support. Individually they all pointed to
HV1:/hosted_engine with backupvol=HV2,HV3...
HV1 lost its bootsector, which was discovered upon a reboot. This had
zero impact, as designed, on the VM's.
However, now that HV1 is down, how does one go about replacing the
original HV? The backup servers point to HV1, and you cannot readd the
HV through the GUI, and the CLI will not readd it as it's already
there... you cannot remove it as it is down in the GUI...
Pointing the other HV's to their own storage may make sense for
multiple instances of the hosted_engine, however it's nice that the
gluster volumes are replicated and that one VM can be relaunched when
a HV error is detected. It's also consuming less resources.
What's the procedure to replace the original VM?
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
1:25 a.m.
This is a multi-part message in MIME format.
--------------1BBBADD28F98A6EF25AE5C4A
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit
Hello Hanson,
Below is the procedure to replace the host with same FQDN where
existing host OS has to be re-installed. If the ovirt version you are
running is 4.0, steps 14 and 15 are not required. You could reinstall
the host from UI with HostedEngine->Deploy option.
*
1.
Move host (host3) to maintenance in UI
2.
Re-install OS, subscribe to channels & install required packages,
prepare bricks (if needed)
3.
Check gluster peer status from working node to obtain UUID of host
being replaced
4.
Create brick directories by running the command mkdir /rhgs/brick{1..3}
5.
Put /etc/fstab entries in the new node by copying it from other nodes.
6.
Run mount -a so that bricks are mounted.
7.
Edit gluster UUID in /var/lib/glusterd/glusterd.info
8.
Copy peer info from a working peer to /var/lib/glusterd/peers
(without the peer info of node being replaced, here host3)
9.
Create and remove a tmp dir at all volume mount points
10.
Run the command setfattr -n trusted.non-existent-key -v abc <mount
point> to set extended attributes and remove the extended attribute
by running the command setfattr -x trusted.non-existent-key <mount
point> at all mount points.
11.
Restart glusterd
12.
Ensure heal is in progress and complete
13.
Edit the host, and fetch fingerprint in Advanced details - as
fingerprint is changed due to reinstallation
14.
Run hosted-engine --deploy --config-append=answers.conf on host3
(Should be seen as additional host setup, provide the host number as
known by other hosts)
15.
hosted-engine deploy fails as the host being installed cannot be
added to the engine with hostname already known error. Reinstalling
from the UI and aborting HE setup seems to fix this. ovirt-ha-agent
and ovirt-ha-broker services had to be started manually
1.
Go to UI and click on reinstall button to reinstall the host.
Reinstalling host might fail due not able to configure
management network.
2.
Go to Network Interfaces tab and click on Setup Host Networks
and assign the networks ovirtmgmt and glusternw to the correct nics.
3.
Wait for sometime for the Node to come up and start
ovirt-ha-agent and ovirt-ha-broker services.
*
Thanks
kasturi.
On 09/20/2016 11:27 AM, knarra wrote:
Hi,
Pad [1] contains the procedure to replace the host with same FQDN
where existing host OS has to be re-installed.
[1] https://paste.fedoraproject.org/431252/47435076/
Thanks
kasturi.
On 09/20/2016 06:27 AM, Hanson wrote:
> Hi Guys,
>
> I encountered an unfortunate circumstance today. Possibly an
> achillies heel.
>
> I have three hypervisors, HV1, HV2, HV3, all running gluster for
> hosted engine support. Individually they all pointed to
> HV1:/hosted_engine with backupvol=HV2,HV3...
>
> HV1 lost its bootsector, which was discovered upon a reboot. This had
> zero impact, as designed, on the VM's.
>
> However, now that HV1 is down, how does one go about replacing the
> original HV? The backup servers point to HV1, and you cannot readd
> the HV through the GUI, and the CLI will not readd it as it's already
> there... you cannot remove it as it is down in the GUI...
>
> Pointing the other HV's to their own storage may make sense for
> multiple instances of the hosted_engine, however it's nice that the
> gluster volumes are replicated and that one VM can be relaunched when
> a HV error is detected. It's also consuming less resources.
>
>
> What's the procedure to replace the original VM?
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--------------1BBBADD28F98A6EF25AE5C4A
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hello Hanson,<br>
<br>
Below is the procedure to replace the host with same FQDN
where existing host OS has to be re-installed. If the ovirt
version you are running is 4.0, steps 14 and 15 are not required.
You could reinstall the host from UI with HostedEngine->Deploy
option.<br>
<br>
<meta http-equiv="content-type" content="text/html;
charset=windows-1252">
<meta http-equiv="content-type" content="text/html;
charset=windows-1252">
<b style="font-weight:normal;"
id="docs-internal-guid-2ce616e6-463c-f821-1898-5ecbdf4b8407">
<ol style="margin-top:0pt;margin-bottom:0pt;">
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Move
host (host3) to maintenance in UI</span></p>
</li>
</ol>
<br>
<ol style="margin-top:0pt;margin-bottom:0pt;"
start="2">
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Re-install
OS, subscribe to channels & install required packages, prepare bricks (if
needed)</span></p>
</li>
</ol>
<br>
<ol style="margin-top:0pt;margin-bottom:0pt;"
start="3">
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Check
gluster peer status from working node to obtain UUID of host being
replaced</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Create
brick directories by running the command mkdir /rhgs/brick{1..3}</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Put
/etc/fstab entries in the new node by copying it from other nodes.</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Run
mount -a so that bricks are mounted.</span></p>
</li>
</ol>
<br>
<ol style="margin-top:0pt;margin-bottom:0pt;"
start="7">
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Edit
gluster UUID in /var/lib/glusterd/glusterd.info </span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Copy
peer info from a working peer to /var/lib/glusterd/peers (without the peer info of node
being replaced, here host3) </span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Create
and remove a tmp dir at all volume mount points </span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
Run the command setfattr -n trusted.non-existent-key -v abc <mount point>
to set extended attributes and remove the extended attribute by running the command
setfattr -x trusted.non-existent-key <mount point> at all mount
points.</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Restart
glusterd</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Ensure
heal is in progress and complete</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Edit
the host, and fetch fingerprint in Advanced details - as fingerprint is changed due to
reinstallation</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Run
hosted-engine --deploy --config-append=answers.conf on host3 (Should be seen as
additional host setup, provide the host number as known by other
hosts)</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:14.666666666666666px;font-family:Arial;color:#ff0000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#ff0000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">hosted-engine
deploy fails as the host being installed cannot be added to the engine with hostname
already known error. Reinstalling from the UI and aborting HE setup seems to fix this.
ovirt-ha-agent and ovirt-ha-broker services had to be started
manually</span></p>
</li>
<ol style="margin-top:0pt;margin-bottom:0pt;">
<li dir="ltr"
style="list-style-type:lower-alpha;font-size:14.666666666666666px;font-family:Arial;color:#ff0000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#ff0000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Go
to UI and click on reinstall button to reinstall the host. Reinstalling host might fail
due not able to configure management network.</span></p>
</li>
<li dir="ltr"
style="list-style-type:lower-alpha;font-size:14.666666666666666px;font-family:Arial;color:#ff0000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#ff0000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Go
to Network Interfaces tab and click on Setup Host Networks and assign the networks
ovirtmgmt and glusternw to the correct nics.</span></p>
</li>
<li dir="ltr"
style="list-style-type:lower-alpha;font-size:14.666666666666666px;font-family:Arial;color:#ff0000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:14.666666666666666px;font-family:Arial;color:#ff0000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Wait
for sometime for the Node to come up and start ovirt-ha-agent and ovirt-ha-broker
services.</span></p>
</li>
</ol>
</ol>
</b>
<meta charset="utf-8">
<br>
Thanks<br>
kasturi.<br>
<br>
On 09/20/2016 11:27 AM, knarra wrote:<br>
</div>
<blockquote
cite="mid:cece5fa2-088c-5b84-8446-449d865dbb3b@redhat.com"
type="cite">Hi,
<br>
<br>
Pad [1] contains the procedure to replace the host with same
FQDN where existing host OS has to be re-installed.
<br>
<br>
[1] <a class="moz-txt-link-freetext"
href="https://paste.fedoraproject.org/431252/47435076/">http...
<br>
<br>
Thanks
<br>
kasturi.
<br>
<br>
On 09/20/2016 06:27 AM, Hanson wrote:
<br>
<blockquote type="cite">Hi Guys,
<br>
<br>
I encountered an unfortunate circumstance today. Possibly an
achillies heel.
<br>
<br>
I have three hypervisors, HV1, HV2, HV3, all running gluster for
hosted engine support. Individually they all pointed to
HV1:/hosted_engine with backupvol=HV2,HV3...
<br>
<br>
HV1 lost its bootsector, which was discovered upon a reboot.
This had zero impact, as designed, on the VM's.
<br>
<br>
However, now that HV1 is down, how does one go about replacing
the original HV? The backup servers point to HV1, and you cannot
readd the HV through the GUI, and the CLI will not readd it as
it's already there... you cannot remove it as it is down in the
GUI...
<br>
<br>
Pointing the other HV's to their own storage may make sense for
multiple instances of the hosted_engine, however it's nice that
the gluster volumes are replicated and that one VM can be
relaunched when a HV error is detected. It's also consuming less
resources.
<br>
<br>
<br>
What's the procedure to replace the original VM?
<br>
<br>
<br>
<br>
_______________________________________________
<br>
Users mailing list
<br>
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<br>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
<br>
</blockquote>
<br>
<br>
_______________________________________________
<br>
Users mailing list
<br>
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<br>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
<br>
</blockquote>
<p><br>
</p>
</body>
</html>
--------------1BBBADD28F98A6EF25AE5C4A--
2987
days inactive
2987
days old
2 comments
2 participants
participants (2)
-
Hanson -
knarra