8:32 p.m.
This is a multi-part message in MIME format.
--------------6DF367159DC0824DA089BFDD
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
On 09/04/2017 06:32 PM, Simone Tiraboschi wrote:
On Mon, Sep 4, 2017 at 6:24 PM, Arsène Gschwind
<arsene.gschwind(a)unibas.ch <mailto:arsene.gschwind@unibas.ch>> wrote:
On 09/04/2017 02:51 PM, Simone Tiraboschi wrote:
>
>
> On Mon, Sep 4, 2017 at 2:21 PM, Arsène Gschwind
> <arsene.gschwind(a)unibas.ch <mailto:arsene.gschwind@unibas.ch>>
wrote:
>
>
>
> On 09/04/2017 02:01 PM, Simone Tiraboschi wrote:
>>
>>
>> On Mon, Sep 4, 2017 at 1:55 PM, Arsène Gschwind
>> <arsene.gschwind(a)unibas.ch
>> <mailto:arsene.gschwind@unibas.ch>> wrote:
>>
>>
>>
>> On 09/04/2017 01:52 PM, Simone Tiraboschi wrote:
>>>
>>>
>>> On Mon, Sep 4, 2017 at 12:23 PM, Arsène Gschwind
>>> <arsene.gschwind(a)unibas.ch
>>> <mailto:arsene.gschwind@unibas.ch>> wrote:
>>>
>>> Hi Simone,
>>>
>>>
>>> On 09/04/2017 11:14 AM, Simone Tiraboschi wrote:
>>>>
>>>>
>>>> On Mon, Sep 4, 2017 at 10:56 AM, Arsène Gschwind
>>>> <arsene.gschwind(a)unibas.ch
>>>> <mailto:arsene.gschwind@unibas.ch>> wrote:
>>>>
>>>> Hi Didi,
>>>>
>>>>
>>>> On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:
>>>>> On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind
>>>>> <arsene.gschwind(a)unibas.ch>
>>>>> <mailto:arsene.gschwind@unibas.ch> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> A while ago I had some problem with
hosted-engine network which wasn't set
>>>>>> correctly at deploy time, so I finally
decided to redeploy the hosted engine
>>>>>> in the hope the network will be set correctly
this time. I've followed this
>>>>>> procedure:
>>>>>>
>>>>>> Stop all VMs
>>>>>> Full backup of HE DB and export to safe
place
>>>>>> Cleanup HE storage
followinghttps://access.redhat.com/solutions/2121581
>>>>>>
<https://access.redhat.com/solutions/2121581>
>>>>>> Reboot Hosts
>>>>>> Re-deploy HE until DB recovery
>>>>>> Recover DB adding the following param:
>>>>>> --he-remove-storage-vm Removes
the hosted-engine storage
>>>>>> domain, all its entities and the
hosted-engine VM during restore.
>>>>>> --he-remove-hosts Removes
all the hosted-engine hosts
>>>>>> during restore.
>>>>>>
>>>>>> Finalize HE deployment.
>>>>>>
>>>>>> Everything did run without errors and I'm
able to access Web UI.
>>>>>>
>>>>>> But now I don't see my HE VM and its
respective Storage Domain, the logs
>>>>>> says it isn't able to import it. I see
all other SD and I'm able to manage
>>>>>> my VMs as before.
>>>>>>
>>>>>> Please find attached engine.log
>>>>> I think this is your problem:
>>>>>
>>>>> 2017-09-04 03:26:14,272+02 INFO
>>>>>
[org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand]
>>>>> (org.ovirt.thread.pool-6-thread-24) [2383eaa0]
There are existing luns
>>>>> in the system which are part of VG id
>>>>> 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'
>>>> I don't see a VG with this ID, here the IDs I
>>>> see on the hosts:
>>>>
>>>> VG #PV #LV #SN Attr VSize VFree
>>>> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0
>>>> wz--n- 99.62g 14.50g
>>>> b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0
>>>> wz--n- 10.00t 2.31t
>>>> b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0
>>>> wz--n- 5.27t 2.50t
>>>>
>>>>
>>>>
>>>> Could you please repeat the command on host
>>>> adm-kvmh70 ?
>>>>
>>>> 2017-09-04 09:04:18,163+02 INFO
>>>> [org.ovirt.engine.core.bll.st
>>>>
<http://org.ovirt.engine.core.bll.st>orage.domain.ImportHostedEngineSto...]
>>>> (org.ovirt.thread.pool-6-thread-34) [247a3718]
>>>> Running command:
>>>> ImportHostedEngineStorageDomainCommand internal: true.
>>>> 2017-09-04 09:04:18,189+02 INFO
>>>>
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand]
>>>> (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2]
>>>> START, GetVGInfoVDSCommand(HostName = adm-kvmh70,
>>>> GetVGInfoVDSCommandParameters:{runAsync='true',
>>>> hostId='acbacabb-6c4a-43fd-a1e2-2d7ff2f6f98b',
>>>> VGID='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'}),
>>>> log id: 6693b98a
>>>> 2017-09-04 09:04:18,232+02 INFO
>>>>
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand]
>>>> (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2]
>>>> FINISH, GetVGInfoVDSCommand, return:
>>>> [LUNs:{id='repl_HostedEngine',
>>>>
physicalVolumeId='kYN8Jj-FBDw-MhxI-XcoZ-w1zH-eQL8-IRIgzO',
>>>>
volumeGroupId='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g',
>>>> serial='SHITACHI_OPEN-V_50488888',
lunMapping='4',
>>>> vendorId='HITACHI', productId='OPEN-V',
>>>> lunConnections='[]', deviceSize='100',
pvSize='0',
>>>> peCount='797', peAllocatedCount='681',
>>>> vendorName='HITACHI',
pathsDictionary='[sdf=true,
>>>> sdu=true, sdk=true, sdp=true]',
>>>> pathsCapacity='[sdf=100, sdu=100, sdk=100,
>>>> sdp=100]', lunType='FCP',
status='null',
>>>> diskId='null', diskAlias='null',
>>>>
storageDomainId='6b62cc06-fc44-4c38-af6d-bfd9cbe73246',
>>>> storageDomainName='null',
>>>> discardMaxSize='268435456',
>>>> discardZeroesData='true'}], log id: 6693b98a
>>>> 2017-09-04 09:04:18,245+02 INFO
>>>> [org.ovirt.engine.core.bll.st
>>>>
<http://org.ovirt.engine.core.bll.st>orage.domain.AddExistingBlockStora...]
>>>> (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2]
>>>> There are existing luns in the system which are
>>>> part of VG id
'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'
>>>> 2017-09-04 09:04:18,245+02 WARN
>>>> [org.ovirt.engine.core.bll.st
>>>>
<http://org.ovirt.engine.core.bll.st>orage.domain.AddExistingBlockStora...]
>>>> (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2]
>>>> Validation of action
>>>> 'AddExistingBlockStorageDomain' failed for user
>>>> SYSTEM. Reasons:
>>>>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ADD,ACTION_TYPE_FAILED_IMPORT_STORAGE_DOMAIN_EXTERNAL_LUN_DISK_EXIST
>>> I don't know which command you are talking about, I
>>> didn't run any command since it tries to import the
>>> SD automatically.
>>>
>>>
>>> Sorry, can you please run vgdisplay on adm-kvmh70 ?
>> No problem! here the result:
>>
>> [root@adm-kvmh70 ~]# vgs
>> VG #PV #LV #SN Attr VSize VFree
>> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n-
>> 99.62g 14.50g
>> b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n-
>> 10.00t 2.31t
>> b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n-
>> 5.27t 2.50t
>> vg_adm-kvmh70 1 3 0 wz--n- 277.90g 218.68g
>>
>>
>> OK, and
>> vgdisplay 6b62cc06-fc44-4c38-af6d-bfd9cbe73246
> [root@adm-kvmh70 ~]# vgdisplay
> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246
> --- Volume group ---
> VG Name 6b62cc06-fc44-4c38-af6d-bfd9cbe73246
> System ID
> Format lvm2
> Metadata Areas 2
> Metadata Sequence No 23
> VG Access read/write
> VG Status resizable
> MAX LV 0
> Cur LV 10
> Open LV 3
> Max PV 0
> Cur PV 1
> Act PV 1
> VG Size 99.62 GiB
> PE Size 128.00 MiB
> Total PE 797
> Alloc PE / Size 681 / 85.12 GiB
> Free PE / Size 116 / 14.50 GiB
> VG UUID vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g
>
>
> OK, vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g is the uuid of the VG
> that engine says that it includes the LUN used by the
> hosted-engine SD as detected by the engine.
> Its name is 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 which is also
> the uuid of that storage domain.
>
> Now we need to understand what's there.
>
> Could you please check if an SD with uuid
> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 is visible in the engine?
The visible SD in the engine
[oVirt shell (connected)]# list storagedomains
id : 072fbaa1-08f3-4a40-9f34-a5ca22dd1d74
name : ovirt-image-repository
description: Public Glance repository for oVirt
id : b2e30961-7cff-4cca-83d6-bee3a4f890ee
name : xxxxxxxxxxx
id : b0414c06-d984-4001-a998-fd9a2e79fb83
name : xxxxxxxxxxx
description: xxxxxxxxxxx
id : 71fa1c61-4a68-469f-9a56-835190d32075
name : xxxxxxxxxxx
id : 9817e64b-fdd9-4d4c-91ee-f3482bb0cc56
name : xxxxxxxxxxxx
description: xxxxxxxxxxxx
> Is adm-kvmh70 involved in hosted-engine? did you redeployed it?
Yes adm-kvmh70 should be a host hosting HE and yes as written
before I did redeploy HE because of a network interface problem. I
did cleanup the lun using the RedHat guide, redeploy HE and
restore the DB using the mentioned parameter to remove HE information.
Could you please set adm-kvmh70 to maintenance mode from the engine
and then reactivate it or better reboot it while in maintenance mode?
I suspect that 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 is just a left
over of the previous hosted-engine storage domain kept open on the host.
I did
reboot adm-kvmh70 while in maintenance mode and also reboot the
engine but it didn't change anything. The import failure is still there...
I've noticed a ovirt-ha-agent error:
ovirt-ha-agent
ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config ERROR
Unable to identify the OVF_STORE volume, falling back to initial
vm.conf. Please ensure you already added your first data domain for
regular VMs
Could that be related?
Thanks
>
> In the past did you tried to manually import the hosted-engine SD
> or something similar?
No, before the "redeployment" the HE SD was auto imported
successfully.
Thanks a lot
>
> thanks
>
> Thanks
>
>> ?
>>
>>
>>>
>>> Thanks,
>>> Arsene
>>>
>>>>
>>>>
>>>>>> Thanks for any help to resolve that issue.
>>>>> I guess you can try to remove this disk/lun from
the engine and let it retry.
>>>> Could let me know how to remove that lun from
>>>> the engine?
>>>>> If the only disk is of the hosted-engine, I guess
it should have been
>>>>> removed by '--he-remove-storage-vm' - if
so, please open a bug
>>>>> describing your flow in detail. Thanks.
>>>> It seems that this option didn't remove the he
>>>> storage information and that this VG is still
>>>> the old one.
>>>>
>>>> Many thanks for your help
>>>> Rgds,
>>>> Arsene
>>>>> Best,
>>>>>
>>>>>> Arsène
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Arsène Gschwind
>>>>>> Fa. Sapify AG im Auftrag der Universität
Basel
>>>>>> IT Services
>>>>>> Klingelbergstr. 70 | CH-4056 Basel |
Switzerland
>>>>>> Tel.+41 79 449 25 63
<tel:+41%2079%20449%2025%2063> |http://its.unibas.ch
>>>>>> ITS-ServiceDesk:support-its@unibas.ch
>>>>>> <mailto:support-its@unibas.ch> |+41 61
267 14 11 <tel:+41%2061%20267%2014%2011>
>>>>>>
>>>>>>
>>>>>>
_______________________________________________
>>>>>> Users mailing list
>>>>>> Users(a)ovirt.org
<mailto:Users@ovirt.org>
>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>
<http://lists.ovirt.org/mailman/listinfo/users>
>>>>>>
>>>>
>>>> --
>>>>
>>>> *Arsène Gschwind*
>>>> Fa. Sapify AG im Auftrag der Universität Basel
>>>> IT Services
>>>> Klingelbergstr. 70 | CH-4056 Basel |
>>>> Switzerland
>>>> Tel. +41 79 449 25 63
>>>> <tel:+41%2079%20449%2025%2063> |
>>>> http://its.unibas.ch <http://its.unibas.ch/>
>>>> ITS-ServiceDesk: support-its(a)unibas.ch
>>>> <mailto:support-its@unibas.ch> | +41 61 267 14
>>>> 11 <tel:+41%2061%20267%2014%2011>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users(a)ovirt.org <mailto:Users@ovirt.org>
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
<http://lists.ovirt.org/mailman/listinfo/users>
>>>>
>>>>
>>>
>>> --
>>>
>>> *Arsène Gschwind*
>>> Fa. Sapify AG im Auftrag der Universität Basel
>>> IT Services
>>> Klingelbergstr. 70 | CH-4056 Basel | Switzerland
>>> Tel. +41 79 449 25 63
>>> <tel:+41%2079%20449%2025%2063> |
>>> http://its.unibas.ch <http://its.unibas.ch/>
>>> ITS-ServiceDesk: support-its(a)unibas.ch
>>> <mailto:support-its@unibas.ch> | +41 61 267 14 11
>>> <tel:+41%2061%20267%2014%2011>
>>>
>>>
>>
>> --
>>
>> *Arsène Gschwind*
>> Fa. Sapify AG im Auftrag der Universität Basel
>> IT Services
>> Klingelbergstr. 70 | CH-4056 Basel | Switzerland
>> Tel. +41 79 449 25 63 <tel:+41%2079%20449%2025%2063> |
>> http://its.unibas.ch <http://its.unibas.ch/>
>> ITS-ServiceDesk: support-its(a)unibas.ch
>> <mailto:support-its@unibas.ch> | +41 61 267 14 11
>> <tel:+41%2061%20267%2014%2011>
>>
>>
>
> --
>
> *Arsène Gschwind*
> Fa. Sapify AG im Auftrag der Universität Basel
> IT Services
> Klingelbergstr. 70 | CH-4056 Basel | Switzerland
> Tel. +41 79 449 25 63 <tel:+41%2079%20449%2025%2063> |
> http://its.unibas.ch <http://its.unibas.ch/>
> ITS-ServiceDesk: support-its(a)unibas.ch
> <mailto:support-its@unibas.ch> | +41 61 267 14 11
> <tel:+41%2061%20267%2014%2011>
>
>
--
*Arsène Gschwind*
Fa. Sapify AG im Auftrag der Universität Basel
IT Services
Klingelbergstr. 70 | CH-4056 Basel | Switzerland
Tel. +41 79 449 25 63 <tel:+41%2079%20449%2025%2063> |
http://its.unibas.ch <http://its.unibas.ch/>
ITS-ServiceDesk: support-its(a)unibas.ch
<mailto:support-its@unibas.ch> | +41 61 267 14 11
<tel:+41%2061%20267%2014%2011>
--
*Arsène Gschwind*
Fa. Sapify AG im Auftrag der Universität Basel
IT Services
Klingelbergstr. 70 | CH-4056 Basel | Switzerland
Tel. +41 79 449 25 63 | http://its.unibas.ch <http://its.unibas.ch/>
ITS-ServiceDesk: support-its(a)unibas.ch | +41 61 267 14 11
--------------6DF367159DC0824DA089BFDD
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 09/04/2017 06:32 PM, Simone
Tiraboschi wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAN8-ONriAHFcwrt1vw9xGidT3-KNez6QqPKNjJj88Y9zy30=8Q@mail.gmail.com">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Sep 4, 2017 at 6:24 PM,
Arsène Gschwind <span dir="ltr"><<a
href="mailto:arsene.gschwind@unibas.ch"
target="_blank"
moz-do-not-send="true">arsene.gschwind(a)unibas.ch</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div>
<div class="gmail-h5">
<p><br>
</p>
<br>
<div
class="gmail-m_2344898123266615019moz-cite-prefix">On
09/04/2017 02:51 PM, Simone Tiraboschi wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Sep 4, 2017
at 2:21 PM, Arsène Gschwind <span
dir="ltr"><<a
href="mailto:arsene.gschwind@unibas.ch"
target="_blank"
moz-do-not-send="true">arsene.gschwind(a)unibas.ch</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div>
<div
class="gmail-m_2344898123266615019gmail-h5">
<p><br>
</p>
<br>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465moz-cite-prefix">On
09/04/2017 02:01 PM, Simone
Tiraboschi wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div
class="gmail_extra"><br>
<div class="gmail_quote">On
Mon, Sep 4, 2017 at 1:55 PM,
Arsène Gschwind <span
dir="ltr"><<a
href="mailto:arsene.gschwind@unibas.ch"
target="_blank"
moz-do-not-send="true">arsene.gschwind(a)unibas.ch</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px
solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-h5">
<p><br>
</p>
<br>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948moz-cite-prefix">On
09/04/2017 01:52
PM, Simone
Tiraboschi wrote:<br>
</div>
<blockquote
type="cite">
<div
dir="ltr"><br>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Mon, Sep 4,
2017 at 12:23
PM, Arsène
Gschwind <span
dir="ltr"><<a
href="mailto:arsene.gschwind@unibas.ch" target="_blank"
moz-do-not-send="true">arsene.gschwind(a)unibas.ch</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<p>Hi Simone,</p>
<div>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-h5">
<br>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079moz-cite-prefix">On
09/04/2017
11:14 AM,
Simone
Tiraboschi
wrote:<br>
</div>
<blockquote
type="cite">
<div
dir="ltr"><br>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Mon, Sep 4,
2017 at 10:56
AM, Arsène
Gschwind <span
dir="ltr"><<a
href="mailto:arsene.gschwind@unibas.ch" target="_blank"
moz-do-not-send="true">arsene.gschwind(a)unibas.ch</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<p>Hi Didi,<br>
</p>
<div>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-h5">
<br>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-cite-prefix">On
09/04/2017
10:15 AM,
Yedidyah Bar
David wrote:<br>
</div>
<blockquote
type="cite">
<pre>On Mon, Sep 4, 2017
at 10:16 AM, Arsène Gschwind
<a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-rfc2396E"
href="mailto:arsene.gschwind@unibas.ch" target="_blank"
moz-do-not-send="true">&lt;arsene.gschwind(a)unibas.ch&gt;</a>
wrote:
</pre>
<blockquote
type="cite">
<pre>Hi all,
A while ago I had some problem with hosted-engine network which wasn't set
correctly at deploy time, so I finally decided to redeploy the hosted engine
in the hope the network will be set correctly this time. I've followed this
procedure:
Stop all VMs
Full backup of HE DB and export to safe place
Cleanup HE storage following <a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-freetext"
href="https://access.redhat.com/solutions/2121581" target="_blank"
moz-do-not-send="true">https://access.redhat.com/solu<wbr...
Reboot Hosts
Re-deploy HE until DB recovery
Recover DB adding the following param:
--he-remove-storage-vm Removes the hosted-engine storage
domain, all its entities and the hosted-engine VM during restore.
--he-remove-hosts Removes all the hosted-engine hosts
during restore.
Finalize HE deployment.
Everything did run without errors and I'm able to access Web UI.
But now I don't see my HE VM and its respective Storage Domain, the logs
says it isn't able to import it. I see all other SD and I'm able to manage
my VMs as before.
Please find attached engine.log
</pre>
</blockquote>
<pre>I think this is your
problem:
2017-09-04 03:26:14,272+02 INFO
[org.ovirt.engine.core.bll.sto<wbr>rage.domain.AddExistingBlockSt<wbr>orageDomainCommand]
(org.ovirt.thread.pool-6-threa<wbr>d-24) [2383eaa0] There are existing luns
in the system which are part of VG id
'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL<wbr>2z-wbTU8g'</pre>
</blockquote>
</div>
</div>
I don't see a
VG with this
ID, here the
IDs I see on
the hosts:<br>
<br>
<tt>
VG <wbr>
#PV #LV #SN
Attr VSize
VFree
</tt><tt><br>
</tt><tt>
6b62cc06-fc44-4c38-af6d-bfd9cb<wbr>e73246
1 10 0
wz--n- 99.62g
14.50g</tt><tt><br>
</tt><tt>
b0414c06-d984-4001-a998-fd9a2e<wbr>79fb83
2 70 0
wz--n- 10.00t
2.31t</tt><tt><br>
</tt><tt>
b2e30961-7cff-4cca-83d6-bee3a4<wbr>f890ee
2 47 0
wz--n- 5.27t
2.50t</tt></div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<div>Could you
please repeat
the command on
host
adm-kvmh70 ?</div>
<div><br>
</div>
<div>
<div>2017-09-04
09:04:18,163+02 INFO [<a href="http://org.ovirt.engine.core.bll.st"
target="_blank"
moz-do-not-send="true">org.ovirt.engine.core.bll.st</a><wbr>orage.domain.ImportHostedEngin<wbr>eStorageDomainCommand]
(org.ovirt.thread.pool-6-threa<wbr>d-34) [247a3718] Running command:
ImportHostedEngineStorageDomai<wbr>nCommand
internal:
true.</div>
<div>2017-09-04
09:04:18,189+02 INFO
[org.ovirt.engine.core.vdsbro<wbr>ker.vdsbroker.GetVGInfoVDSComm<wbr>and]
(org.ovirt.thread.pool-6-threa<wbr>d-34) [7d2e6cb2] START,
GetVGInfoVDSCommand(HostName
= adm-kvmh70,
GetVGInfoVDSCommandParameters:<wbr>{runAsync='true',
hostId='acbacabb-6c4a-43fd-a1e<wbr>2-2d7ff2f6f98b',
VGID='vvIoS2-fZTZ-99Ox-Ltzq-pr<wbr>8U-SL2z-wbTU8g'}),
log id:
6693b98a</div>
<div>2017-09-04
09:04:18,232+02 INFO
[org.ovirt.engine.core.vdsbro<wbr>ker.vdsbroker.GetVGInfoVDSComm<wbr>and]
(org.ovirt.thread.pool-6-threa<wbr>d-34) [7d2e6cb2] FINISH,
GetVGInfoVDSCommand,
return:
[LUNs:{id='repl_HostedEngine',
physicalVolumeId='kYN8Jj-FBDw-<wbr>MhxI-XcoZ-w1zH-eQL8-IRIgzO',
volumeGroupId='vvIoS2-fZTZ-99O<wbr>x-Ltzq-pr8U-SL2z-wbTU8g',
serial='SHITACHI_OPEN-V_504888<wbr>88', lunMapping='4',
vendorId='HITACHI',
productId='OPEN-V', lunConnections='[]', deviceSize='100',
pvSize='0',
peCount='797',
peAllocatedCount='681', vendorName='HITACHI',
pathsDictionary='[sdf=true,
sdu=true,
sdk=true,
sdp=true]',
pathsCapacity='[sdf=100,
sdu=100,
sdk=100,
sdp=100]',
lunType='FCP',
status='null',
diskId='null',
diskAlias='null',
storageDomainId='6b62cc06-fc44<wbr>-4c38-af6d-bfd9cbe73246',
storageDomainName='null', discardMaxSize='268435456',
discardZeroesData='true'}],
log id:
6693b98a</div>
<div>2017-09-04
09:04:18,245+02 INFO [<a href="http://org.ovirt.engine.core.bll.st"
target="_blank"
moz-do-not-send="true">org.ovirt.engine.core.bll.st</a><wbr>orage.domain.AddExistingBlockS<wbr>torageDomainCommand]
(org.ovirt.thread.pool-6-threa<wbr>d-34) [7d2e6cb2] There are existing
luns in the
system which
are part of VG
id
'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL<wbr>2z-wbTU8g'</div>
<div>2017-09-04
09:04:18,245+02 WARN [<a href="http://org.ovirt.engine.core.bll.st"
target="_blank"
moz-do-not-send="true">org.ovirt.engine.core.bll.st</a><wbr>orage.domain.AddExistingBlockS<wbr>torageDomainCommand]
(org.ovirt.thread.pool-6-threa<wbr>d-34) [7d2e6cb2] Validation of action
'AddExistingBlockStorageDomain<wbr>' failed for user SYSTEM. Reasons:
VAR__TYPE__STORAGE__DOMAIN,VAR<wbr>__ACTION__ADD,ACTION_TYPE_FAIL<wbr>ED_IMPORT_STORAGE_DOMAIN_EXTER<wbr>NAL_LUN_DISK_EXIST</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
I don't know
which command
you are
talking about,
I didn't run
any command
since it tries
to import the
SD
automatically.<br>
</div>
</blockquote>
<div><br>
</div>
<div>Sorry,
can you please
run vgdisplay
on adm-kvmh70
?</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
No problem! here the
result:<br>
<br>
<tt>[root@adm-kvmh70 ~]#
vgs</tt><span
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-"><tt><br>
</tt><tt>
VG <wbr>
#PV #LV #SN Attr
VSize VFree
</tt><tt><br>
</tt><tt>
6b62cc06-fc44-4c38-af6d-bfd9cb<wbr>e73246
1 10 0 wz--n-
99.62g
14.50g</tt><tt><br>
</tt><tt>
b0414c06-d984-4001-a998-fd9a2e<wbr>79fb83
2 70 0 wz--n-
10.00t
2.31t</tt><tt><br>
</tt><tt>
b2e30961-7cff-4cca-83d6-bee3a4<wbr>f890ee
2 47 0 wz--n-
5.27t
2.50t</tt><tt><br>
</tt></span><tt>
vg_adm-kvmh70 <wbr>
1 3 0 wz--n-
277.90g 218.68g</tt>
<div>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-h5"><tt><br>
</tt></div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>OK, and </div>
<div> vgdisplay
6b62cc06-fc44-4c38-af6d-bfd9cb<wbr>e73246</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
<tt>[root@adm-kvmh70 ~]# vgdisplay
6b62cc06-fc44-4c38-af6d-bfd9cb<wbr>e73246</tt><tt><br>
</tt><tt> --- Volume group
---</tt><tt><br>
</tt><tt> VG Name
6b62cc06-fc44-4c38-af6d-bfd9cb<wbr>e73246</tt><tt><br>
</tt><tt> System ID
</tt><tt><br>
</tt><tt> Format
lvm2</tt><tt><br>
</tt><tt> Metadata Areas
2</tt><tt><br>
</tt><tt> Metadata Sequence No
23</tt><tt><br>
</tt><tt> VG Access
read/write</tt><tt><br>
</tt><tt> VG Status
resizable</tt><tt><br>
</tt><tt> MAX LV
0</tt><tt><br>
</tt><tt> Cur LV
10</tt><tt><br>
</tt><tt> Open LV
3</tt><tt><br>
</tt><tt> Max PV
0</tt><tt><br>
</tt><tt> Cur PV
1</tt><tt><br>
</tt><tt> Act PV
1</tt><tt><br>
</tt><tt> VG Size 99.62
GiB</tt><tt><br>
</tt><tt> PE Size 128.00
MiB</tt><tt><br>
</tt><tt> Total PE
797</tt><tt><br>
</tt><tt> Alloc PE / Size 681 /
85.12 GiB</tt><tt><br>
</tt><tt> Free PE / Size 116 /
14.50 GiB</tt><tt><br>
</tt><tt> VG UUID
vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2<wbr>z-wbTU8g<br>
</tt></div>
</blockquote>
<div><br>
</div>
<div>OK, vvIoS2-fZTZ-99Ox-Ltzq-<wbr>pr8U-SL2z-wbTU8g
is the uuid of the VG that engine says
that it includes the LUN used by the
hosted-engine SD as detected by the
engine.</div>
<div>Its name
is 6b62cc06-fc44-4c38-af6d-<wbr>bfd9cbe73246
which is also the uuid of that storage
domain.</div>
<div><br>
</div>
<div>Now we need to understand what's
there.</div>
<div><br>
</div>
<div>Could you please check if an SD with
uuid 6b62cc06-fc44-4c38-af6d-<wbr>bfd9cbe73246
is visible in the engine?</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
The visible SD in the engine<br>
<tt><br>
[oVirt shell (connected)]# list storagedomains
</tt><tt><br>
</tt><tt><br>
</tt><tt>id :
072fbaa1-08f3-4a40-9f34-<wbr>a5ca22dd1d74</tt><tt><br>
</tt><tt>name :
ovirt-image-repository</tt><tt><br>
</tt><tt>description: Public Glance repository for
oVirt</tt><tt><br>
</tt><tt><br>
</tt><tt>id :
b2e30961-7cff-4cca-83d6-<wbr>bee3a4f890ee</tt><tt><br>
</tt><tt>name :
xxxxxxxxxxx</tt><tt><br>
</tt><tt><br>
</tt><tt>id :
b0414c06-d984-4001-a998-<wbr>fd9a2e79fb83</tt><tt><br>
</tt><tt>name :
xxxxxxxxxxx</tt><tt><br>
</tt><tt>description:
xxxxxxxxxxx</tt><tt><br>
</tt><tt><br>
</tt><tt>id :
71fa1c61-4a68-469f-9a56-<wbr>835190d32075</tt><tt><br>
</tt><tt>name :
xxxxxxxxxxx</tt><tt><br>
</tt><tt><br>
</tt><tt>id :
9817e64b-fdd9-4d4c-91ee-<wbr>f3482bb0cc56</tt><tt><br>
</tt><tt>name :
xxxxxxxxxxxx</tt><tt><br>
</tt><tt>description: xxxxxxxxxxxx</tt><span
class="gmail-"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>Is adm-kvmh70 involved in hosted-engine?
did you redeployed it?</div>
</div>
</div>
</div>
</blockquote>
</span> Yes adm-kvmh70 should be a host hosting HE and
yes as written before I did redeploy HE because of a
network interface problem. I did cleanup the lun using
the RedHat guide, redeploy HE and restore the DB using
the mentioned parameter to remove HE information.</div>
</blockquote>
<div><br>
</div>
<div>Could you please set adm-kvmh70 to maintenance mode
from the engine and then reactivate it or better reboot it
while in maintenance mode?</div>
<div>I suspect that 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 is
just a left over of the previous hosted-engine storage
domain kept open on the host. <br>
</div>
</div>
</div>
</div>
</blockquote>
I did reboot adm-kvmh70 while in maintenance mode and also reboot
the engine but it didn't change anything. The import failure is
still there...<br>
I've noticed a ovirt-ha-agent error:<br>
<tt><br>
ovirt-ha-agent
ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config
ERROR Unable to identify the OVF_STORE volume, falling back to
initial vm.conf. Please ensure you already added your first data
domain for regular VMs<br>
<br>
</tt>Could that be related?<br>
Thanks<br>
<tt></tt>
<blockquote type="cite"
cite="mid:CAN8-ONriAHFcwrt1vw9xGidT3-KNez6QqPKNjJj88Y9zy30=8Q@mail.gmail.com">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span
class="gmail-"><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>In the past did you tried to manually
import the hosted-engine SD or something
similar?</div>
</div>
</div>
</div>
</blockquote>
</span> No, before the "redeployment" the HE SD was auto
imported successfully.<br>
<br>
Thanks a lot
<div>
<div class="gmail-h5"><br>
<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>thanks</div>
</div>
</div>
</div>
</blockquote>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><tt>
<br>
Thanks<br>
</tt>
<div>
<div
class="gmail-m_2344898123266615019gmail-h5"><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>?</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px
solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-h5"><tt>
</tt><br>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<br>
Thanks,<br>
Arsene
<div>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-h5"><br>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF"><span
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-"><tt><br>
</tt><br>
<blockquote
type="cite">
<blockquote
type="cite">
<pre>Thanks for any help
to resolve that issue.
</pre>
</blockquote>
<pre>I guess you can try
to remove this disk/lun from the engine and let it retry.</pre>
</blockquote>
</span> Could
let me know
how to remove
that lun from
the engine?<span
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-"><br>
<blockquote
type="cite">
<pre>If the only disk is
of the hosted-engine, I guess it should have been
removed by '--he-remove-storage-vm' - if so, please open a bug
describing your flow in detail. Thanks.</pre>
</blockquote>
</span> It
seems that
this option
didn't remove
the he storage
information
and that this
VG is still
the old one.<br>
<br>
Many thanks
for your help<br>
Rgds,<br>
Arsene<span
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-"><br>
<blockquote
type="cite">
<pre>Best,
</pre>
<blockquote
type="cite">
<pre>Arsène
--
Arsène Gschwind
Fa. Sapify AG im Auftrag der Universität Basel
IT Services
Klingelbergstr. 70 | CH-4056 Basel | Switzerland
Tel. <a href="tel:+41%2079%20449%2025%2063" value="+41794492563"
target="_blank" moz-do-not-send="true">+41 79 449 25 63</a>
| <a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-freetext"
href="http://its.unibas.ch" target="_blank"
moz-do-not-send="true">http://its.unibas.ch</a>
ITS-ServiceDesk: <a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-abbreviated"
href="mailto:support-its@unibas.ch" target="_blank"
moz-do-not-send="true">support-its(a)unibas.ch</a> | <a
href="tel:+41%2061%20267%2014%2011" value="+41612671411"
target="_blank" moz-do-not-send="true">+41 61 267 14 11</a>
______________________________<wbr>_________________
Users mailing list
<a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org" target="_blank"
moz-do-not-send="true">Users(a)ovirt.org</a>
<a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank"
moz-do-not-send="true">http://lists.ovirt.org/mailman<wbr...
</pre>
</blockquote>
</blockquote>
<br>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-signature">--
<br>
<p
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039western"
style="margin-bottom:0in;line-height:150%"> <font
color="#000000"><font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> <b>Arsène Gschwind</b>
</font> </font>
<font
color="#000000">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> </font> </font>
</font> <font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> </font> </font>
<font face="Tahoma,
serif"> </font>
<font
color="#000000">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> <br>
</font> </font>
</font> <font
color="#7f7f7f"> <font face="Tahoma, serif"> <font
style="font-size:8pt"
size="1"> Fa.
Sapify AG im
Auftrag der
Universität
Basel<br>
IT Services<br>
Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br>
Tel. <a
href="tel:+41%2079%20449%2025%2063"
value="+41794492563" target="_blank"
moz-do-not-send="true">+41 79 449
25 63</a> |
</font> </font>
</font> <a
href="http://its.unibas.ch/"
target="_blank" moz-do-not-send="true"> <font face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> http://its.unibas.ch </font>
</font> </a><br>
<font
color="#7f7f7f">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> ITS-ServiceDesk: <a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-abbreviated"
href="mailto:support-its@unibas.ch" target="_blank"
moz-do-not-send="true">support-its(a)unibas.ch</a>
| <a
href="tel:+41%2061%20267%2014%2011"
value="+41612671411" target="_blank"
moz-do-not-send="true">+41 61 267
14 11</a> </font>
</font> </font>
</font></p>
<font
color="#000000">
</font></div>
</span></div>
<br>
______________________________<wbr>_________________<br>
Users mailing
list<br>
<a
href="mailto:Users@ovirt.org"
target="_blank"
moz-do-not-send="true">Users(a)ovirt.org</a><br>
<a
href="http://lists.ovirt.org/mailman/listinfo/users"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.ovirt.org/mailman<wbr...
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079moz-signature">--
<br>
<p
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079western"
style="margin-bottom:0in;line-height:150%"> <font
color="#000000"><font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> <b>Arsène Gschwind</b>
</font> </font>
<font
color="#000000">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> </font> </font>
</font> <font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> </font> </font>
<font face="Tahoma,
serif"> </font>
<font
color="#000000">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> <br>
</font> </font>
</font> <font
color="#7f7f7f"> <font face="Tahoma, serif"> <font
style="font-size:8pt"
size="1"> Fa.
Sapify AG im
Auftrag der
Universität
Basel<br>
IT Services<br>
Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br>
Tel. <a
href="tel:+41%2079%20449%2025%2063"
value="+41794492563" target="_blank"
moz-do-not-send="true">+41 79 449
25 63</a> |
</font> </font>
</font> <a
href="http://its.unibas.ch/"
target="_blank" moz-do-not-send="true"> <font face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> http://its.unibas.ch </font>
</font> </a><br>
<font
color="#7f7f7f">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> ITS-ServiceDesk: <a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948gmail-m_-6999624817399485079moz-txt-link-abbreviated"
href="mailto:support-its@unibas.ch" target="_blank"
moz-do-not-send="true">support-its(a)unibas.ch</a>
| <a
href="tel:+41%2061%20267%2014%2011"
value="+41612671411" target="_blank"
moz-do-not-send="true">+41 61 267
14 11</a> </font>
</font> </font>
</font></p>
<font
color="#000000">
</font></div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948moz-signature">--
<br>
<p
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948western"
style="margin-bottom:0in;line-height:150%"> <font
color="#000000"><font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> <b>Arsène Gschwind</b>
</font> </font>
<font
color="#000000">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> </font> </font>
</font> <font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> </font> </font>
<font face="Tahoma,
serif"> </font>
<font
color="#000000">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> <br>
</font> </font>
</font> <font
color="#7f7f7f"> <font face="Tahoma, serif"> <font
style="font-size:8pt"
size="1"> Fa.
Sapify AG im
Auftrag der
Universität
Basel<br>
IT Services<br>
Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br>
Tel. <a
href="tel:+41%2079%20449%2025%2063"
value="+41794492563" target="_blank"
moz-do-not-send="true">+41 79 449
25 63</a> |
</font> </font>
</font> <a
href="http://its.unibas.ch/"
target="_blank" moz-do-not-send="true"> <font face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> http://its.unibas.ch </font>
</font> </a><br>
<font
color="#7f7f7f">
<font
face="Tahoma,
serif"> <font
style="font-size:8pt" size="1"> ITS-ServiceDesk: <a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465gmail-m_-5209736991554863948moz-txt-link-abbreviated"
href="mailto:support-its@unibas.ch" target="_blank"
moz-do-not-send="true">support-its(a)unibas.ch</a>
| <a
href="tel:+41%2061%20267%2014%2011"
value="+41612671411" target="_blank"
moz-do-not-send="true">+41 61 267
14 11</a> </font>
</font> </font>
</font></p>
<font
color="#000000">
</font></div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<div
class="gmail-m_2344898123266615019gmail-m_1688827238525709465moz-signature">--
<br>
<p
class="gmail-m_2344898123266615019gmail-m_1688827238525709465western"
style="margin-bottom:0in;line-height:150%"> <font
color="#000000"><font
face="Tahoma, serif"> <font
style="font-size:8pt"
size="1"> <b>Arsène
Gschwind</b> </font>
</font>
<font color="#000000"> <font
face="Tahoma, serif"> <font
style="font-size:8pt"
size="1"> </font>
</font>
</font> <font face="Tahoma,
serif"> <font
style="font-size:8pt"
size="1"> </font>
</font>
<font face="Tahoma, serif">
</font>
<font color="#000000"> <font
face="Tahoma, serif"> <font
style="font-size:8pt"
size="1"> <br>
</font> </font> </font>
<font color="#7f7f7f"> <font
face="Tahoma, serif"> <font
style="font-size:8pt"
size="1"> Fa. Sapify AG
im Auftrag der
Universität Basel<br>
IT Services<br>
Klingelbergstr. 70 |
CH-4056 Basel |
Switzerland<br>
Tel. <a
href="tel:+41%2079%20449%2025%2063"
value="+41794492563"
target="_blank"
moz-do-not-send="true">+41
79 449 25 63</a> |
</font>
</font> </font> <a
href="http://its.unibas.ch/"
target="_blank"
moz-do-not-send="true"> <font
face="Tahoma, serif"> <font
style="font-size:8pt"
size="1">
http://its.unibas.ch </font>
</font> </a><br>
<font color="#7f7f7f"> <font
face="Tahoma, serif"> <font
style="font-size:8pt"
size="1">
ITS-ServiceDesk: <a
class="gmail-m_2344898123266615019gmail-m_1688827238525709465moz-txt-link-abbreviated"
href="mailto:support-its@unibas.ch" target="_blank"
moz-do-not-send="true">support-its(a)unibas.ch</a>
| <a
href="tel:+41%2061%20267%2014%2011"
value="+41612671411"
target="_blank"
moz-do-not-send="true">+41
61 267 14 11</a> </font>
</font> </font>
</font></p>
<font color="#000000">
</font></div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<div
class="gmail-m_2344898123266615019moz-signature">--
<br>
<p class="gmail-m_2344898123266615019western"
style="margin-bottom:0in;line-height:150%"> <font
color="#000000"><font face="Tahoma,
serif"> <font
style="font-size:8pt" size="1">
<b>Arsène
Gschwind</b> </font> </font> <font
color="#000000"> <font face="Tahoma,
serif">
<font style="font-size:8pt"
size="1"> </font>
</font> </font> <font face="Tahoma,
serif">
<font style="font-size:8pt"
size="1"> </font>
</font> <font face="Tahoma, serif">
</font>
<font color="#000000"> <font
face="Tahoma,
serif"> <font style="font-size:8pt"
size="1"> <br>
</font> </font> </font> <font
color="#7f7f7f"> <font face="Tahoma,
serif">
<font style="font-size:8pt"
size="1"> Fa.
Sapify AG im Auftrag der Universität
Basel<br>
IT Services<br>
Klingelbergstr. 70 | CH-4056 Basel |
Switzerland<br>
Tel. <a
href="tel:+41%2079%20449%2025%2063"
value="+41794492563"
target="_blank"
moz-do-not-send="true">+41 79 449 25
63</a> | </font> </font>
</font>
<a href="http://its.unibas.ch/"
target="_blank" moz-do-not-send="true">
<font
face="Tahoma, serif"> <font
style="font-size:8pt" size="1">
http://its.unibas.ch </font> </font>
</a><br>
<font color="#7f7f7f"> <font
face="Tahoma,
serif"> <font style="font-size:8pt"
size="1"> ITS-ServiceDesk: <a
class="gmail-m_2344898123266615019moz-txt-link-abbreviated"
href="mailto:support-its@unibas.ch"
target="_blank"
moz-do-not-send="true">support-its(a)unibas.ch</a>
| <a
href="tel:+41%2061%20267%2014%2011"
value="+41612671411"
target="_blank"
moz-do-not-send="true">+41 61 267 14
11</a> </font> </font> </font>
</font></p>
<font color="#000000"> </font></div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<p class="western" style="margin-bottom: 0in; line-height:
150%">
<font color="#000000"><font face="Tahoma, serif">
<font
style="font-size: 8pt" size="1"> <b>Arsène
Gschwind</b> </font>
</font>
<font color="#000000"> <font face="Tahoma,
serif"> <font
style="font-size: 8pt" size="1"> </font>
</font>
</font>
<font face="Tahoma, serif"> <font style="font-size:
8pt"
size="1"> </font>
</font>
<font face="Tahoma, serif">
</font>
<font color="#000000"> <font face="Tahoma,
serif"> <font
style="font-size: 8pt" size="1"> <br>
</font> </font>
</font>
<font color="#7f7f7f"> <font face="Tahoma,
serif"> <font
style="font-size: 8pt" size="1"> Fa. Sapify AG im
Auftrag der Universität Basel<br>
IT Services<br>
Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br>
Tel. +41 79 449 25 63 | </font> </font>
</font>
<a href="http://its.unibas.ch/"> <font face="Tahoma,
serif">
<font style="font-size: 8pt" size="1">
http://its.unibas.ch </font> </font>
</a><br>
<font color="#7f7f7f"> <font face="Tahoma,
serif"> <font
style="font-size: 8pt" size="1"> ITS-ServiceDesk:
<a class="moz-txt-link-abbreviated"
href="mailto:support-its@unibas.ch">support-its@unibas.ch</a> | +41 61
267 14 11 </font> </font>
</font>
</font></p>
<font color="#000000">
</font></div>
</body>
</html>
--------------6DF367159DC0824DA089BFDD--