4:17 p.m.
On Wed, May 5, 2021 at 3:18 PM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote:
Status Update:
During migration between Host1 and Host3, links are being created and I see no Errors as
before in the agent/broker logs.
Having all links in place, and having waited 24hours, for potential Engine updates, I
tried to deploy HE on Host2:
Part1:
From oVirt UI – HostedEngine Deploy
Imeddiately on Host2 started seeing messages like “ Is the HostedEngine deployed ?”
hosted-engine.conf got populated only with host_id and ca_path
it failed
Part2:
I noticed that during deployment no links in
/var/rund/vdsm/storage/<hosted_storage_id> were created
Copied hosted-engine.conf from Host1 to Host2
Replace the host_id with correct value
Reran the deployment
Noticed that /var/rund/vdsm/storage/<hosted_storage_id> two links got created, one
of them being the link to metadata_image
Host1 and Host3 hosted-engine –vm-status was showing Host2 but with status unknown/stale
data
Deployment failed
Part3:
Since link to conf_image was not created in first phase, I added it manually
Populated the hosted-engine.conf
Reran the deployment
Same result
Hosted-engine.conf on Host2 would end up with only host_id and ca_path values and
deployment would fail
At this point, I cleaned up all hosted-engie remains from Host2 using
ovirt-hosted-engine-cleanup and removed metadata of Host2 from host1 and Host3
I am out of ideas. It seems that Host1 and Host3 are happily operating, but I am unable
to add any other hosts to HE pool.
Please assist if you have any ideas.
Look, Marko - I admit it seems to me like you simply enjoy debugging
this yourself...
1. Did you try to reinstall the OS on host2? If not, is there any
reason not to? Other than the (legitimate!) wish to "understand what's
broken and fix just that"? Would reinstallation take a lot of
time/work? You can also do a full backup beforehand and then compare
later, to see what the differences were.
2. It's very hard to help you by only guessing around. If you have a
concrete issue, such as "I add a host with 'Deploy Hosted Engine' and
this fails", then please provide all relevant logs.
3. If you still want to continue debugging by yourself, fine - the
code is open, and at least I personally try quite hard to make it
easy, in the relatively small parts of the code I touched, to search
around it even without having a complete picture of its structure,
mainly by making texts inside logs be "unique enough" so that you can
easily find them in the code.
I am open to executing restore – but I would much rather like to at least discover where
or what is the problem, before moving to planning restore.
Good luck and best regards,
--
Didi
4:31 p.m.
New subject: Unable to migrate Engine to another HE Host
Hi Yedidyah,
Thank you for such honest answer.
Debugging gives me better insight into how platform operates, and the documentation rarely
dives this deep. I did learn a lot but my intent was not to debug on my own.
I sure did re-install OS of Host2 and not only once (thanks to Foreman) it’s easier and
faster than expected.
I also got help from my neighbor Strahil, who is always happy to help, but I could not get
to the bottom of what is wrong, in order to see if its fixable or not.
2. It's very hard to help you by only guessing around. If you have a
concrete issue, such as "I add a host with 'Deploy Hosted Engine' and
this fails", then please provide all relevant logs.
> the list of problems I reported came due to power outage, so it
was hard to know where to start.
> I will open new issue with focus on problem with adding new HA
Host and we take it from there.
3. If you still want to continue debugging by yourself, fine - the
code is open, and at least I personally try quite hard to make it
easy, in the relatively small parts of the code I touched, to search
around it even without having a complete picture of its structure,
mainly by making texts inside logs be "unique enough" so that you can
easily find them in the code.
> I am proud of oVirt / I am running two production platforms, for
almost three years, upgraded several times, in two regions with 1000+ tenants at any
moment
> but each time I have an issue, it takes me a while to get used to where to look and
how to search
Thank you both Strahil and Yedidyah .
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic@activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The
Netherlands. The information contained in this message may be legally privileged and
confidential. It is intended to be read only by the individual or entity to whom it is
addressed or by their designee. If the reader of this message is not the intended
recipient, you are on notice that any distribution of this message, in any form, is
strictly prohibited. If you have received this message in error, please immediately
notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and
delete or destroy any copy of this message.
From: Yedidyah Bar David <didi(a)redhat.com>
Date: Wednesday, 5 May 2021 at 15:18
To: Marko Vrgotic <M.Vrgotic(a)activevideo.com>
Cc: Strahil Nikolov <hunter86_bg(a)yahoo.com>, users(a)ovirt.org
<users(a)ovirt.org>
Subject: Re: [ovirt-users] Re: Unable to migrate Engine to another HE Host
***CAUTION: This email originated from outside of the organization. Do not click links or
open attachments unless you recognize the sender!!!***
On Wed, May 5, 2021 at 3:18 PM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote:
Status Update:
During migration between Host1 and Host3, links are being created and I see no Errors as
before in the agent/broker logs.
Having all links in place, and having waited 24hours, for potential Engine updates, I
tried to deploy HE on Host2:
Part1:
From oVirt UI – HostedEngine Deploy
Imeddiately on Host2 started seeing messages like “ Is the HostedEngine deployed ?”
hosted-engine.conf got populated only with host_id and ca_path
it failed
Part2:
I noticed that during deployment no links in
/var/rund/vdsm/storage/<hosted_storage_id> were created
Copied hosted-engine.conf from Host1 to Host2
Replace the host_id with correct value
Reran the deployment
Noticed that /var/rund/vdsm/storage/<hosted_storage_id> two links got created, one
of them being the link to metadata_image
Host1 and Host3 hosted-engine –vm-status was showing Host2 but with status unknown/stale
data
Deployment failed
Part3:
Since link to conf_image was not created in first phase, I added it manually
Populated the hosted-engine.conf
Reran the deployment
Same result
Hosted-engine.conf on Host2 would end up with only host_id and ca_path values and
deployment would fail
At this point, I cleaned up all hosted-engie remains from Host2 using
ovirt-hosted-engine-cleanup and removed metadata of Host2 from host1 and Host3
I am out of ideas. It seems that Host1 and Host3 are happily operating, but I am unable
to add any other hosts to HE pool.
Please assist if you have any ideas.
Look, Marko - I admit it seems to me like you simply enjoy debugging
this yourself...
1. Did you try to reinstall the OS on host2? If not, is there any
reason not to? Other than the (legitimate!) wish to "understand what's
broken and fix just that"? Would reinstallation take a lot of
time/work? You can also do a full backup beforehand and then compare
later, to see what the differences were.
2. It's very hard to help you by only guessing around. If you have a
concrete issue, such as "I add a host with 'Deploy Hosted Engine' and
this fails", then please provide all relevant logs.
3. If you still want to continue debugging by yourself, fine - the
code is open, and at least I personally try quite hard to make it
easy, in the relatively small parts of the code I touched, to search
around it even without having a complete picture of its structure,
mainly by making texts inside logs be "unique enough" so that you can
easily find them in the code.
I am open to executing restore – but I would much rather like to at least discover where
or what is the problem, before moving to planning restore.
Good luck and best regards,
--
Didi
1298
days inactive
1298
days old
1 comments
2 participants
participants (2)
-
Marko Vrgotic -
Yedidyah Bar David