Failed to import the Hosted Engine Storage Domain
Hi all, A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure: 1. Stop all VMs 2. Full backup of HE DB and export to safe place 3. Cleanup HE storage following https://access.redhat.com/solutions/2121581 4. Reboot Hosts 5. Re-deploy HE until DB recovery 6. Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore. 7. Finalize HE deployment. Everything did run without errors and I'm able to access Web UI. But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before. Please find attached engine.log Thanks for any help to resolve that issue. Arsène -- *Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 | http://its.unibas.ch <http://its.unibas.ch/> ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11
On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind <arsene.gschwind@unibas.ch> wrote:
Hi all,
A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure:
Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage following https://access.redhat.com/solutions/2121581 Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore.
Finalize HE deployment.
Everything did run without errors and I'm able to access Web UI.
But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before.
Please find attached engine.log
I think this is your problem: 2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'
Thanks for any help to resolve that issue.
I guess you can try to remove this disk/lun from the engine and let it retry. If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks. Best,
Arsène
--
Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 | http://its.unibas.ch ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Didi
This is a multi-part message in MIME format. --------------3FF162BE98B9974DA61F11B6 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi Didi, On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:
On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind <arsene.gschwind@unibas.ch> wrote:
Hi all,
A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure:
Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage following https://access.redhat.com/solutions/2121581 Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore.
Finalize HE deployment.
Everything did run without errors and I'm able to access Web UI.
But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before.
Please find attached engine.log I think this is your problem:
2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g' I don't see a VG with this ID, here the IDs I see on the hosts:
Thanks for any help to resolve that issue.
I guess you can try to remove this disk/lun from the engine and let it retry. Could let me know how to remove that lun from the engine?
If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks. It seems that this option didn't remove the he storage information and
VG #PV #LV #SN Attr VSize VFree 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t that this VG is still the old one. Many thanks for your help Rgds, Arsene
Best,
Arsène
--
Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 | http://its.unibas.ch ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- *Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 | http://its.unibas.ch <http://its.unibas.ch/> ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11 --------------3FF162BE98B9974DA61F11B6 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <p>Hi Didi,<br> </p> <br> <div class="moz-cite-prefix">On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:<br> </div> <blockquote type="cite" cite="mid:CAHRwYXsr+_6d6uaS447DJLrdb7vC1Bm1iBcQ=b490HcgcFtbnw@mail.gmail.com"> <pre wrap="">On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind <a class="moz-txt-link-rfc2396E" href="mailto:arsene.gschwind@unibas.ch"><arsene.gschwind@unibas.ch></a> wrote: </pre> <blockquote type="cite"> <pre wrap="">Hi all, A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure: Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage following <a class="moz-txt-link-freetext" href="https://access.redhat.com/solutions/2121581">https://access.redhat.com/solutions/2121581</a> Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore. Finalize HE deployment. Everything did run without errors and I'm able to access Web UI. But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before. Please find attached engine.log </pre> </blockquote> <pre wrap=""> I think this is your problem: 2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'</pre> </blockquote> I don't see a VG with this ID, here the IDs I see on the hosts:<br> <br> <tt> VG #PV #LV #SN Attr VSize VFree </tt><tt><br> </tt><tt> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g</tt><tt><br> </tt><tt> b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t</tt><tt><br> </tt><tt> b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t</tt><tt><br> </tt><br> <blockquote type="cite" cite="mid:CAHRwYXsr+_6d6uaS447DJLrdb7vC1Bm1iBcQ=b490HcgcFtbnw@mail.gmail.com"> <pre wrap=""> </pre> <blockquote type="cite"> <pre wrap=""> Thanks for any help to resolve that issue. </pre> </blockquote> <pre wrap=""> I guess you can try to remove this disk/lun from the engine and let it retry.</pre> </blockquote> Could let me know how to remove that lun from the engine?<br> <blockquote type="cite" cite="mid:CAHRwYXsr+_6d6uaS447DJLrdb7vC1Bm1iBcQ=b490HcgcFtbnw@mail.gmail.com"> <pre wrap=""> If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks.</pre> </blockquote> It seems that this option didn't remove the he storage information and that this VG is still the old one.<br> <br> Many thanks for your help<br> Rgds,<br> Arsene<br> <blockquote type="cite" cite="mid:CAHRwYXsr+_6d6uaS447DJLrdb7vC1Bm1iBcQ=b490HcgcFtbnw@mail.gmail.com"> <pre wrap=""> Best, </pre> <blockquote type="cite"> <pre wrap=""> Arsène -- Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 | <a class="moz-txt-link-freetext" href="http://its.unibas.ch">http://its.unibas.ch</a> ITS-ServiceDesk: <a class="moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch">support-its@unibas.ch</a> | +41 61 267 14 11 _______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <pre wrap=""> </pre> </blockquote> <br> <div class="moz-signature">-- <br> <p class="western" style="margin-bottom: 0in; line-height: 150%"> <font color="#000000"><font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> <b>Arsène Gschwind</b> </font> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> </font> </font> </font> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> </font> </font> <font face="Tahoma, serif"> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> <br> </font> </font> </font> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> Fa. Sapify AG im Auftrag der Universität Basel<br> IT Services<br> Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br> Tel. +41 79 449 25 63 | </font> </font> </font> <a href="http://its.unibas.ch/"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> http://its.unibas.ch </font> </font> </a><br> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> ITS-ServiceDesk: <a class="moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch">support-its@unibas.ch</a> | +41 61 267 14 11 </font> </font> </font> </font></p> <font color="#000000"> </font></div> </body> </html> --------------3FF162BE98B9974DA61F11B6--
On Mon, Sep 4, 2017 at 10:56 AM, Arsène Gschwind <arsene.gschwind@unibas.ch> wrote:
Hi Didi,
On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:
On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind<arsene.gschwind@unibas.ch> <arsene.gschwind@unibas.ch> wrote:
Hi all,
A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure:
Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage following https://access.redhat.com/solutions/2121581 Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore.
Finalize HE deployment.
Everything did run without errors and I'm able to access Web UI.
But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before.
Please find attached engine.log
I think this is your problem:
2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'
I don't see a VG with this ID, here the IDs I see on the hosts:
VG #PV #LV #SN Attr VSize VFree 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t
Could you please repeat the command on host adm-kvmh70 ? 2017-09-04 09:04:18,163+02 INFO [org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [247a3718] Running command: ImportHostedEngineStorageDomainCommand internal: true. 2017-09-04 09:04:18,189+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] START, GetVGInfoVDSCommand(HostName = adm-kvmh70, GetVGInfoVDSCommandParameters:{runAsync='true', hostId='acbacabb-6c4a-43fd-a1e2-2d7ff2f6f98b', VGID='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'}), log id: 6693b98a 2017-09-04 09:04:18,232+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] FINISH, GetVGInfoVDSCommand, return: [LUNs:{id='repl_HostedEngine', physicalVolumeId='kYN8Jj-FBDw-MhxI-XcoZ-w1zH-eQL8-IRIgzO', volumeGroupId='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g', serial='SHITACHI_OPEN-V_50488888', lunMapping='4', vendorId='HITACHI', productId='OPEN-V', lunConnections='[]', deviceSize='100', pvSize='0', peCount='797', peAllocatedCount='681', vendorName='HITACHI', pathsDictionary='[sdf=true, sdu=true, sdk=true, sdp=true]', pathsCapacity='[sdf=100, sdu=100, sdk=100, sdp=100]', lunType='FCP', status='null', diskId='null', diskAlias='null', storageDomainId='6b62cc06-fc44-4c38-af6d-bfd9cbe73246', storageDomainName='null', discardMaxSize='268435456', discardZeroesData='true'}], log id: 6693b98a 2017-09-04 09:04:18,245+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g' 2017-09-04 09:04:18,245+02 WARN [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] Validation of action 'AddExistingBlockStorageDomain' failed for user SYSTEM. Reasons: VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ADD,ACTION_TYPE_FAILED_IMPORT_STORAGE_DOMAIN_EXTERNAL_LUN_DISK_EXIST
Thanks for any help to resolve that issue.
I guess you can try to remove this disk/lun from the engine and let it retry.
Could let me know how to remove that lun from the engine?
If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks.
It seems that this option didn't remove the he storage information and that this VG is still the old one.
Many thanks for your help Rgds, Arsene
Best,
Arsène
--
Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11 <+41%2061%20267%2014%2011>
_______________________________________________ Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
--
*Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11 <+41%2061%20267%2014%2011>
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
This is a multi-part message in MIME format. --------------A4238539BE9451880B1B7F6D Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi Simone, On 09/04/2017 11:14 AM, Simone Tiraboschi wrote:
On Mon, Sep 4, 2017 at 10:56 AM, Arsène Gschwind <arsene.gschwind@unibas.ch <mailto:arsene.gschwind@unibas.ch>> wrote:
Hi Didi,
On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:
On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind <arsene.gschwind@unibas.ch> <mailto:arsene.gschwind@unibas.ch> wrote:
Hi all,
A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure:
Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage followinghttps://access.redhat.com/solutions/2121581 <https://access.redhat.com/solutions/2121581> Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore.
Finalize HE deployment.
Everything did run without errors and I'm able to access Web UI.
But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before.
Please find attached engine.log
I think this is your problem:
2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'
I don't see a VG with this ID, here the IDs I see on the hosts:
VG #PV #LV #SN Attr VSize VFree 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t
Could you please repeat the command on host adm-kvmh70 ?
2017-09-04 09:04:18,163+02 INFO [org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [247a3718] Running command: ImportHostedEngineStorageDomainCommand internal: true. 2017-09-04 09:04:18,189+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] START, GetVGInfoVDSCommand(HostName = adm-kvmh70, GetVGInfoVDSCommandParameters:{runAsync='true', hostId='acbacabb-6c4a-43fd-a1e2-2d7ff2f6f98b', VGID='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'}), log id: 6693b98a 2017-09-04 09:04:18,232+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] FINISH, GetVGInfoVDSCommand, return: [LUNs:{id='repl_HostedEngine', physicalVolumeId='kYN8Jj-FBDw-MhxI-XcoZ-w1zH-eQL8-IRIgzO', volumeGroupId='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g', serial='SHITACHI_OPEN-V_50488888', lunMapping='4', vendorId='HITACHI', productId='OPEN-V', lunConnections='[]', deviceSize='100', pvSize='0', peCount='797', peAllocatedCount='681', vendorName='HITACHI', pathsDictionary='[sdf=true, sdu=true, sdk=true, sdp=true]', pathsCapacity='[sdf=100, sdu=100, sdk=100, sdp=100]', lunType='FCP', status='null', diskId='null', diskAlias='null', storageDomainId='6b62cc06-fc44-4c38-af6d-bfd9cbe73246', storageDomainName='null', discardMaxSize='268435456', discardZeroesData='true'}], log id: 6693b98a 2017-09-04 09:04:18,245+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g' 2017-09-04 09:04:18,245+02 WARN [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] Validation of action 'AddExistingBlockStorageDomain' failed for user SYSTEM. Reasons: VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ADD,ACTION_TYPE_FAILED_IMPORT_STORAGE_DOMAIN_EXTERNAL_LUN_DISK_EXIST
I don't know which command you are talking about, I didn't run any command since it tries to import the SD automatically. Thanks, Arsene
Thanks for any help to resolve that issue.
I guess you can try to remove this disk/lun from the engine and let it retry. Could let me know how to remove that lun from the engine? If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks. It seems that this option didn't remove the he storage information and that this VG is still the old one.
Many thanks for your help Rgds, Arsene
Best,
Arsène
--
Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel.+41 79 449 25 63 <tel:+41%2079%20449%2025%2063> |http://its.unibas.ch ITS-ServiceDesk:support-its@unibas.ch <mailto:support-its@unibas.ch> |+41 61 267 14 11 <tel:+41%2061%20267%2014%2011>
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>
--
*Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 <tel:+41%2079%20449%2025%2063> | http://its.unibas.ch <http://its.unibas.ch/> ITS-ServiceDesk: support-its@unibas.ch <mailto:support-its@unibas.ch> | +41 61 267 14 11 <tel:+41%2061%20267%2014%2011>
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>
-- *Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 | http://its.unibas.ch <http://its.unibas.ch/> ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11 --------------A4238539BE9451880B1B7F6D Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <p>Hi Simone,</p> <br> <div class="moz-cite-prefix">On 09/04/2017 11:14 AM, Simone Tiraboschi wrote:<br> </div> <blockquote type="cite" cite="mid:CAN8-ONo4AYMuyU+4YAQYvND_K2+Qc-WRES1WJ0hSJ7=MQ7-GUg@mail.gmail.com"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <div dir="ltr"><br> <div class="gmail_extra"><br> <div class="gmail_quote">On Mon, Sep 4, 2017 at 10:56 AM, Arsène Gschwind <span dir="ltr"><<a href="mailto:arsene.gschwind@unibas.ch" target="_blank" moz-do-not-send="true">arsene.gschwind@unibas.ch</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF"> <p>Hi Didi,<br> </p> <div> <div class="gmail-h5"> <br> <div class="gmail-m_-1388566559526568039moz-cite-prefix">On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:<br> </div> <blockquote type="cite"> <pre>On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind <a class="gmail-m_-1388566559526568039moz-txt-link-rfc2396E" href="mailto:arsene.gschwind@unibas.ch" target="_blank" moz-do-not-send="true"><arsene.gschwind@unibas.ch></a> wrote: </pre> <blockquote type="cite"> <pre>Hi all, A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure: Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage following <a class="gmail-m_-1388566559526568039moz-txt-link-freetext" href="https://access.redhat.com/solutions/2121581" target="_blank" moz-do-not-send="true">https://access.redhat.com/<wbr>solutions/2121581</a> Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore. Finalize HE deployment. Everything did run without errors and I'm able to access Web UI. But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before. Please find attached engine.log </pre> </blockquote> <pre>I think this is your problem: 2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>AddExistingBlockStorageDomainC<wbr>ommand] (org.ovirt.thread.pool-6-<wbr>thread-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-<wbr>SL2z-wbTU8g'</pre> </blockquote> </div> </div> I don't see a VG with this ID, here the IDs I see on the hosts:<br> <br> <tt> VG <wbr> #PV #LV #SN Attr VSize VFree </tt><tt><br> </tt><tt> 6b62cc06-fc44-4c38-af6d-<wbr>bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g</tt><tt><br> </tt><tt> b0414c06-d984-4001-a998-<wbr>fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t</tt><tt><br> </tt><tt> b2e30961-7cff-4cca-83d6-<wbr>bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t</tt></div> </blockquote> <div><br> </div> <div><br> </div> <div>Could you please repeat the command on host adm-kvmh70 ?</div> <div><br> </div> <div> <div>2017-09-04 09:04:18,163+02 INFO [org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [247a3718] Running command: ImportHostedEngineStorageDomainCommand internal: true.</div> <div>2017-09-04 09:04:18,189+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] START, GetVGInfoVDSCommand(HostName = adm-kvmh70, GetVGInfoVDSCommandParameters:{runAsync='true', hostId='acbacabb-6c4a-43fd-a1e2-2d7ff2f6f98b', VGID='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'}), log id: 6693b98a</div> <div>2017-09-04 09:04:18,232+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] FINISH, GetVGInfoVDSCommand, return: [LUNs:{id='repl_HostedEngine', physicalVolumeId='kYN8Jj-FBDw-MhxI-XcoZ-w1zH-eQL8-IRIgzO', volumeGroupId='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g', serial='SHITACHI_OPEN-V_50488888', lunMapping='4', vendorId='HITACHI', productId='OPEN-V', lunConnections='[]', deviceSize='100', pvSize='0', peCount='797', peAllocatedCount='681', vendorName='HITACHI', pathsDictionary='[sdf=true, sdu=true, sdk=true, sdp=true]', pathsCapacity='[sdf=100, sdu=100, sdk=100, sdp=100]', lunType='FCP', status='null', diskId='null', diskAlias='null', storageDomainId='6b62cc06-fc44-4c38-af6d-bfd9cbe73246', storageDomainName='null', discardMaxSize='268435456', discardZeroesData='true'}], log id: 6693b98a</div> <div>2017-09-04 09:04:18,245+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'</div> <div>2017-09-04 09:04:18,245+02 WARN [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] Validation of action 'AddExistingBlockStorageDomain' failed for user SYSTEM. Reasons: VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ADD,ACTION_TYPE_FAILED_IMPORT_STORAGE_DOMAIN_EXTERNAL_LUN_DISK_EXIST</div> </div> </div> </div> </div> </blockquote> I don't know which command you are talking about, I didn't run any command since it tries to import the SD automatically.<br> <br> Thanks,<br> Arsene<br> <blockquote type="cite" cite="mid:CAN8-ONo4AYMuyU+4YAQYvND_K2+Qc-WRES1WJ0hSJ7=MQ7-GUg@mail.gmail.com"> <div dir="ltr"> <div class="gmail_extra"> <div class="gmail_quote"> <div> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF"><span class="gmail-"><tt><br> </tt><br> <blockquote type="cite"> <blockquote type="cite"> <pre>Thanks for any help to resolve that issue. </pre> </blockquote> <pre>I guess you can try to remove this disk/lun from the engine and let it retry.</pre> </blockquote> </span> Could let me know how to remove that lun from the engine?<span class="gmail-"><br> <blockquote type="cite"> <pre>If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks.</pre> </blockquote> </span> It seems that this option didn't remove the he storage information and that this VG is still the old one.<br> <br> Many thanks for your help<br> Rgds,<br> Arsene<span class="gmail-"><br> <blockquote type="cite"> <pre>Best, </pre> <blockquote type="cite"> <pre>Arsène -- Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. <a href="tel:+41%2079%20449%2025%2063" value="+41794492563" target="_blank" moz-do-not-send="true">+41 79 449 25 63</a> | <a class="gmail-m_-1388566559526568039moz-txt-link-freetext" href="http://its.unibas.ch" target="_blank" moz-do-not-send="true">http://its.unibas.ch</a> ITS-ServiceDesk: <a class="gmail-m_-1388566559526568039moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch" target="_blank" moz-do-not-send="true">support-its@unibas.ch</a> | <a href="tel:+41%2061%20267%2014%2011" value="+41612671411" target="_blank" moz-do-not-send="true">+41 61 267 14 11</a> ______________________________<wbr>_________________ Users mailing list <a class="gmail-m_-1388566559526568039moz-txt-link-abbreviated" href="mailto:Users@ovirt.org" target="_blank" moz-do-not-send="true">Users@ovirt.org</a> <a class="gmail-m_-1388566559526568039moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" moz-do-not-send="true">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a> </pre> </blockquote> </blockquote> <br> <div class="gmail-m_-1388566559526568039moz-signature">-- <br> <p class="gmail-m_-1388566559526568039western" style="margin-bottom:0in;line-height:150%"> <font color="#000000"><font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> <b>Arsène Gschwind</b> </font> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> </font> </font> </font> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> </font> </font> <font face="Tahoma, serif"> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> <br> </font> </font> </font> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> Fa. Sapify AG im Auftrag der Universität Basel<br> IT Services<br> Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br> Tel. <a href="tel:+41%2079%20449%2025%2063" value="+41794492563" target="_blank" moz-do-not-send="true">+41 79 449 25 63</a> | </font> </font> </font> <a href="http://its.unibas.ch/" target="_blank" moz-do-not-send="true"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> http://its.unibas.ch </font> </font> </a><br> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> ITS-ServiceDesk: <a class="gmail-m_-1388566559526568039moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch" target="_blank" moz-do-not-send="true">support-its@unibas.ch</a> | <a href="tel:+41%2061%20267%2014%2011" value="+41612671411" target="_blank" moz-do-not-send="true">+41 61 267 14 11</a> </font> </font> </font> </font></p> <font color="#000000"> </font></div> </span></div> <br> ______________________________<wbr>_________________<br> Users mailing list<br> <a href="mailto:Users@ovirt.org" moz-do-not-send="true">Users@ovirt.org</a><br> <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br> <br> </blockquote> </div> <br> </div> </div> </blockquote> <br> <div class="moz-signature">-- <br> <p class="western" style="margin-bottom: 0in; line-height: 150%"> <font color="#000000"><font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> <b>Arsène Gschwind</b> </font> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> </font> </font> </font> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> </font> </font> <font face="Tahoma, serif"> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> <br> </font> </font> </font> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> Fa. Sapify AG im Auftrag der Universität Basel<br> IT Services<br> Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br> Tel. +41 79 449 25 63 | </font> </font> </font> <a href="http://its.unibas.ch/"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> http://its.unibas.ch </font> </font> </a><br> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> ITS-ServiceDesk: <a class="moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch">support-its@unibas.ch</a> | +41 61 267 14 11 </font> </font> </font> </font></p> <font color="#000000"> </font></div> </body> </html> --------------A4238539BE9451880B1B7F6D--
On Mon, Sep 4, 2017 at 12:23 PM, Arsène Gschwind <arsene.gschwind@unibas.ch> wrote:
Hi Simone,
On 09/04/2017 11:14 AM, Simone Tiraboschi wrote:
On Mon, Sep 4, 2017 at 10:56 AM, Arsène Gschwind < arsene.gschwind@unibas.ch> wrote:
Hi Didi,
On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:
On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind<arsene.gschwind@unibas.ch> <arsene.gschwind@unibas.ch> wrote:
Hi all,
A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure:
Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage following https://access.redhat.com/solutions/2121581 Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore.
Finalize HE deployment.
Everything did run without errors and I'm able to access Web UI.
But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before.
Please find attached engine.log
I think this is your problem:
2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'
I don't see a VG with this ID, here the IDs I see on the hosts:
VG #PV #LV #SN Attr VSize VFree 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t
Could you please repeat the command on host adm-kvmh70 ?
2017-09-04 09:04:18,163+02 INFO [org.ovirt.engine.core.bll. storage.domain.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [247a3718] Running command: ImportHostedEngineStorageDomainCommand internal: true. 2017-09-04 09:04:18,189+02 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] START, GetVGInfoVDSCommand(HostName = adm-kvmh70, GetVGInfoVDSCommandParameters:{runAsync='true', hostId='acbacabb-6c4a-43fd-a1e2-2d7ff2f6f98b', VGID='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'}), log id: 6693b98a 2017-09-04 09:04:18,232+02 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] FINISH, GetVGInfoVDSCommand, return: [LUNs:{id='repl_HostedEngine', physicalVolumeId='kYN8Jj-FBDw-MhxI-XcoZ-w1zH-eQL8-IRIgzO', volumeGroupId='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g', serial='SHITACHI_OPEN-V_50488888', lunMapping='4', vendorId='HITACHI', productId='OPEN-V', lunConnections='[]', deviceSize='100', pvSize='0', peCount='797', peAllocatedCount='681', vendorName='HITACHI', pathsDictionary='[sdf=true, sdu=true, sdk=true, sdp=true]', pathsCapacity='[sdf=100, sdu=100, sdk=100, sdp=100]', lunType='FCP', status='null', diskId='null', diskAlias='null', storageDomainId='6b62cc06-fc44-4c38-af6d-bfd9cbe73246', storageDomainName='null', discardMaxSize='268435456', discardZeroesData='true'}], log id: 6693b98a 2017-09-04 09:04:18,245+02 INFO [org.ovirt.engine.core.bll. storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U- SL2z-wbTU8g' 2017-09-04 09:04:18,245+02 WARN [org.ovirt.engine.core.bll. storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] Validation of action 'AddExistingBlockStorageDomain' failed for user SYSTEM. Reasons: VAR__TYPE__STORAGE__DOMAIN, VAR__ACTION__ADD,ACTION_TYPE_FAILED_IMPORT_STORAGE_DOMAIN_ EXTERNAL_LUN_DISK_EXIST
I don't know which command you are talking about, I didn't run any command since it tries to import the SD automatically.
Sorry, can you please run vgdisplay on adm-kvmh70 ?
Thanks, Arsene
Thanks for any help to resolve that issue.
I guess you can try to remove this disk/lun from the engine and let it retry.
Could let me know how to remove that lun from the engine?
If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks.
It seems that this option didn't remove the he storage information and that this VG is still the old one.
Many thanks for your help Rgds, Arsene
Best,
Arsène
--
Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11 <+41%2061%20267%2014%2011>
_______________________________________________ Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
--
*Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch
ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11 <+41%2061%20267%2014%2011>
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--
*Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11 <+41%2061%20267%2014%2011>
This is a multi-part message in MIME format. --------------404D1197B599BE7464062905 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 09/04/2017 01:52 PM, Simone Tiraboschi wrote:
On Mon, Sep 4, 2017 at 12:23 PM, Arsène Gschwind <arsene.gschwind@unibas.ch <mailto:arsene.gschwind@unibas.ch>> wrote:
Hi Simone,
On 09/04/2017 11:14 AM, Simone Tiraboschi wrote:
On Mon, Sep 4, 2017 at 10:56 AM, Arsène Gschwind <arsene.gschwind@unibas.ch <mailto:arsene.gschwind@unibas.ch>> wrote:
Hi Didi,
On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:
On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind <arsene.gschwind@unibas.ch> <mailto:arsene.gschwind@unibas.ch> wrote:
Hi all,
A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure:
Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage followinghttps://access.redhat.com/solutions/2121581 <https://access.redhat.com/solutions/2121581> Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore.
Finalize HE deployment.
Everything did run without errors and I'm able to access Web UI.
But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before.
Please find attached engine.log
I think this is your problem:
2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'
I don't see a VG with this ID, here the IDs I see on the hosts:
VG #PV #LV #SN Attr VSize VFree 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t
Could you please repeat the command on host adm-kvmh70 ?
2017-09-04 09:04:18,163+02 INFO [org.ovirt.engine.core.bll.storage.domain.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [247a3718] Running command: ImportHostedEngineStorageDomainCommand internal: true. 2017-09-04 09:04:18,189+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] START, GetVGInfoVDSCommand(HostName = adm-kvmh70, GetVGInfoVDSCommandParameters:{runAsync='true', hostId='acbacabb-6c4a-43fd-a1e2-2d7ff2f6f98b', VGID='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'}), log id: 6693b98a 2017-09-04 09:04:18,232+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] FINISH, GetVGInfoVDSCommand, return: [LUNs:{id='repl_HostedEngine', physicalVolumeId='kYN8Jj-FBDw-MhxI-XcoZ-w1zH-eQL8-IRIgzO', volumeGroupId='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g', serial='SHITACHI_OPEN-V_50488888', lunMapping='4', vendorId='HITACHI', productId='OPEN-V', lunConnections='[]', deviceSize='100', pvSize='0', peCount='797', peAllocatedCount='681', vendorName='HITACHI', pathsDictionary='[sdf=true, sdu=true, sdk=true, sdp=true]', pathsCapacity='[sdf=100, sdu=100, sdk=100, sdp=100]', lunType='FCP', status='null', diskId='null', diskAlias='null', storageDomainId='6b62cc06-fc44-4c38-af6d-bfd9cbe73246', storageDomainName='null', discardMaxSize='268435456', discardZeroesData='true'}], log id: 6693b98a 2017-09-04 09:04:18,245+02 INFO [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g' 2017-09-04 09:04:18,245+02 WARN [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] Validation of action 'AddExistingBlockStorageDomain' failed for user SYSTEM. Reasons: VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ADD,ACTION_TYPE_FAILED_IMPORT_STORAGE_DOMAIN_EXTERNAL_LUN_DISK_EXIST
I don't know which command you are talking about, I didn't run any command since it tries to import the SD automatically.
Sorry, can you please run vgdisplay on adm-kvmh70 ?
No problem! here the result: [root@adm-kvmh70 ~]# vgs VG #PV #LV #SN Attr VSize VFree 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t vg_adm-kvmh70 1 3 0 wz--n- 277.90g 218.68g
Thanks, Arsene
Thanks for any help to resolve that issue.
I guess you can try to remove this disk/lun from the engine and let it retry. Could let me know how to remove that lun from the engine? If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks. It seems that this option didn't remove the he storage information and that this VG is still the old one.
Many thanks for your help Rgds, Arsene
Best,
Arsène
--
Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel.+41 79 449 25 63 <tel:+41%2079%20449%2025%2063> |http://its.unibas.ch ITS-ServiceDesk:support-its@unibas.ch <mailto:support-its@unibas.ch> |+41 61 267 14 11 <tel:+41%2061%20267%2014%2011>
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>
--
*Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 <tel:+41%2079%20449%2025%2063> | http://its.unibas.ch <http://its.unibas.ch/> ITS-ServiceDesk: support-its@unibas.ch <mailto:support-its@unibas.ch> | +41 61 267 14 11 <tel:+41%2061%20267%2014%2011>
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>
--
*Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 <tel:+41%2079%20449%2025%2063> | http://its.unibas.ch <http://its.unibas.ch/> ITS-ServiceDesk: support-its@unibas.ch <mailto:support-its@unibas.ch> | +41 61 267 14 11 <tel:+41%2061%20267%2014%2011>
-- *Arsène Gschwind* Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. +41 79 449 25 63 | http://its.unibas.ch <http://its.unibas.ch/> ITS-ServiceDesk: support-its@unibas.ch | +41 61 267 14 11 --------------404D1197B599BE7464062905 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <p><br> </p> <br> <div class="moz-cite-prefix">On 09/04/2017 01:52 PM, Simone Tiraboschi wrote:<br> </div> <blockquote type="cite" cite="mid:CAN8-ONrt104F_Q_0ZkqFqLmizpFFS+0njDZisKu5BPtLnDUYEw@mail.gmail.com"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <div dir="ltr"><br> <div class="gmail_extra"><br> <div class="gmail_quote">On Mon, Sep 4, 2017 at 12:23 PM, Arsène Gschwind <span dir="ltr"><<a href="mailto:arsene.gschwind@unibas.ch" target="_blank" moz-do-not-send="true">arsene.gschwind@unibas.ch</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF"> <p>Hi Simone,</p> <div> <div class="gmail-h5"> <br> <div class="gmail-m_-6999624817399485079moz-cite-prefix">On 09/04/2017 11:14 AM, Simone Tiraboschi wrote:<br> </div> <blockquote type="cite"> <div dir="ltr"><br> <div class="gmail_extra"><br> <div class="gmail_quote">On Mon, Sep 4, 2017 at 10:56 AM, Arsène Gschwind <span dir="ltr"><<a href="mailto:arsene.gschwind@unibas.ch" target="_blank" moz-do-not-send="true">arsene.gschwind@unibas.ch</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF"> <p>Hi Didi,<br> </p> <div> <div class="gmail-m_-6999624817399485079gmail-h5"> <br> <div class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-cite-prefix">On 09/04/2017 10:15 AM, Yedidyah Bar David wrote:<br> </div> <blockquote type="cite"> <pre>On Mon, Sep 4, 2017 at 10:16 AM, Arsène Gschwind <a class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-rfc2396E" href="mailto:arsene.gschwind@unibas.ch" target="_blank" moz-do-not-send="true"><arsene.gschwind@unibas.ch></a> wrote: </pre> <blockquote type="cite"> <pre>Hi all, A while ago I had some problem with hosted-engine network which wasn't set correctly at deploy time, so I finally decided to redeploy the hosted engine in the hope the network will be set correctly this time. I've followed this procedure: Stop all VMs Full backup of HE DB and export to safe place Cleanup HE storage following <a class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-freetext" href="https://access.redhat.com/solutions/2121581" target="_blank" moz-do-not-send="true">https://access.redhat.com/solu<wbr>tions/2121581</a> Reboot Hosts Re-deploy HE until DB recovery Recover DB adding the following param: --he-remove-storage-vm Removes the hosted-engine storage domain, all its entities and the hosted-engine VM during restore. --he-remove-hosts Removes all the hosted-engine hosts during restore. Finalize HE deployment. Everything did run without errors and I'm able to access Web UI. But now I don't see my HE VM and its respective Storage Domain, the logs says it isn't able to import it. I see all other SD and I'm able to manage my VMs as before. Please find attached engine.log </pre> </blockquote> <pre>I think this is your problem: 2017-09-04 03:26:14,272+02 INFO [org.ovirt.engine.core.bll.sto<wbr>rage.domain.AddExistingBlockSt<wbr>orageDomainCommand] (org.ovirt.thread.pool-6-threa<wbr>d-24) [2383eaa0] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL<wbr>2z-wbTU8g'</pre> </blockquote> </div> </div> I don't see a VG with this ID, here the IDs I see on the hosts:<br> <br> <tt> VG <wbr> #PV #LV #SN Attr VSize VFree </tt><tt><br> </tt><tt> 6b62cc06-fc44-4c38-af6d-bfd9cb<wbr>e73246 1 10 0 wz--n- 99.62g 14.50g</tt><tt><br> </tt><tt> b0414c06-d984-4001-a998-fd9a2e<wbr>79fb83 2 70 0 wz--n- 10.00t 2.31t</tt><tt><br> </tt><tt> b2e30961-7cff-4cca-83d6-bee3a4<wbr>f890ee 2 47 0 wz--n- 5.27t 2.50t</tt></div> </blockquote> <div><br> </div> <div><br> </div> <div>Could you please repeat the command on host adm-kvmh70 ?</div> <div><br> </div> <div> <div>2017-09-04 09:04:18,163+02 INFO [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>ImportHostedEngineStorageDomai<wbr>nCommand] (org.ovirt.thread.pool-6-<wbr>thread-34) [247a3718] Running command: ImportHostedEngineStorageDomai<wbr>nCommand internal: true.</div> <div>2017-09-04 09:04:18,189+02 INFO [org.ovirt.engine.core.<wbr>vdsbroker.vdsbroker.<wbr>GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-<wbr>thread-34) [7d2e6cb2] START, GetVGInfoVDSCommand(HostName = adm-kvmh70, GetVGInfoVDSCommandParameters:<wbr>{runAsync='true', hostId='acbacabb-6c4a-43fd-<wbr>a1e2-2d7ff2f6f98b', VGID='vvIoS2-fZTZ-99Ox-Ltzq-<wbr>pr8U-SL2z-wbTU8g'}), log id: 6693b98a</div> <div>2017-09-04 09:04:18,232+02 INFO [org.ovirt.engine.core.<wbr>vdsbroker.vdsbroker.<wbr>GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-<wbr>thread-34) [7d2e6cb2] FINISH, GetVGInfoVDSCommand, return: [LUNs:{id='repl_HostedEngine', physicalVolumeId='kYN8Jj-FBDw-<wbr>MhxI-XcoZ-w1zH-eQL8-IRIgzO', volumeGroupId='vvIoS2-fZTZ-<wbr>99Ox-Ltzq-pr8U-SL2z-wbTU8g', serial='SHITACHI_OPEN-V_<wbr>50488888', lunMapping='4', vendorId='HITACHI', productId='OPEN-V', lunConnections='[]', deviceSize='100', pvSize='0', peCount='797', peAllocatedCount='681', vendorName='HITACHI', pathsDictionary='[sdf=true, sdu=true, sdk=true, sdp=true]', pathsCapacity='[sdf=100, sdu=100, sdk=100, sdp=100]', lunType='FCP', status='null', diskId='null', diskAlias='null', storageDomainId='6b62cc06-<wbr>fc44-4c38-af6d-bfd9cbe73246', storageDomainName='null', discardMaxSize='268435456', discardZeroesData='true'}], log id: 6693b98a</div> <div>2017-09-04 09:04:18,245+02 INFO [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>AddExistingBlockStorageDomainC<wbr>ommand] (org.ovirt.thread.pool-6-<wbr>thread-34) [7d2e6cb2] There are existing luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-<wbr>SL2z-wbTU8g'</div> <div>2017-09-04 09:04:18,245+02 WARN [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>AddExistingBlockStorageDomainC<wbr>ommand] (org.ovirt.thread.pool-6-<wbr>thread-34) [7d2e6cb2] Validation of action '<wbr>AddExistingBlockStorageDomain' failed for user SYSTEM. Reasons: VAR__TYPE__STORAGE__DOMAIN,<wbr>VAR__ACTION__ADD,ACTION_TYPE_<wbr>FAILED_IMPORT_STORAGE_DOMAIN_<wbr>EXTERNAL_LUN_DISK_EXIST</div> </div> </div> </div> </div> </blockquote> </div> </div> I don't know which command you are talking about, I didn't run any command since it tries to import the SD automatically.<br> </div> </blockquote> <div><br> </div> <div>Sorry, can you please run vgdisplay on adm-kvmh70 ?</div> </div> </div> </div> </blockquote> No problem! here the result:<br> <br> <tt>[root@adm-kvmh70 ~]# vgs</tt><tt><br> </tt><tt> VG #PV #LV #SN Attr VSize VFree </tt><tt><br> </tt><tt> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g 14.50g</tt><tt><br> </tt><tt> b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t 2.31t</tt><tt><br> </tt><tt> b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t 2.50t</tt><tt><br> </tt><tt> vg_adm-kvmh70 1 3 0 wz--n- 277.90g 218.68g</tt><tt><br> </tt><br> <blockquote type="cite" cite="mid:CAN8-ONrt104F_Q_0ZkqFqLmizpFFS+0njDZisKu5BPtLnDUYEw@mail.gmail.com"> <div dir="ltr"> <div class="gmail_extra"> <div class="gmail_quote"> <div> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF"> <br> Thanks,<br> Arsene <div> <div class="gmail-h5"><br> <blockquote type="cite"> <div dir="ltr"> <div class="gmail_extra"> <div class="gmail_quote"> <div> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF"><span class="gmail-m_-6999624817399485079gmail-"><tt><br> </tt><br> <blockquote type="cite"> <blockquote type="cite"> <pre>Thanks for any help to resolve that issue. </pre> </blockquote> <pre>I guess you can try to remove this disk/lun from the engine and let it retry.</pre> </blockquote> </span> Could let me know how to remove that lun from the engine?<span class="gmail-m_-6999624817399485079gmail-"><br> <blockquote type="cite"> <pre>If the only disk is of the hosted-engine, I guess it should have been removed by '--he-remove-storage-vm' - if so, please open a bug describing your flow in detail. Thanks.</pre> </blockquote> </span> It seems that this option didn't remove the he storage information and that this VG is still the old one.<br> <br> Many thanks for your help<br> Rgds,<br> Arsene<span class="gmail-m_-6999624817399485079gmail-"><br> <blockquote type="cite"> <pre>Best, </pre> <blockquote type="cite"> <pre>Arsène -- Arsène Gschwind Fa. Sapify AG im Auftrag der Universität Basel IT Services Klingelbergstr. 70 | CH-4056 Basel | Switzerland Tel. <a href="tel:+41%2079%20449%2025%2063" value="+41794492563" target="_blank" moz-do-not-send="true">+41 79 449 25 63</a> | <a class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-freetext" href="http://its.unibas.ch" target="_blank" moz-do-not-send="true">http://its.unibas.ch</a> ITS-ServiceDesk: <a class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch" target="_blank" moz-do-not-send="true">support-its@unibas.ch</a> | <a href="tel:+41%2061%20267%2014%2011" value="+41612671411" target="_blank" moz-do-not-send="true">+41 61 267 14 11</a> ______________________________<wbr>_________________ Users mailing list <a class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-abbreviated" href="mailto:Users@ovirt.org" target="_blank" moz-do-not-send="true">Users@ovirt.org</a> <a class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" moz-do-not-send="true">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a> </pre> </blockquote> </blockquote> <br> <div class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-signature">-- <br> <p class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039western" style="margin-bottom:0in;line-height:150%"> <font color="#000000"><font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> <b>Arsène Gschwind</b> </font> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> </font> </font> </font> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> </font> </font> <font face="Tahoma, serif"> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> <br> </font> </font> </font> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> Fa. Sapify AG im Auftrag der Universität Basel<br> IT Services<br> Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br> Tel. <a href="tel:+41%2079%20449%2025%2063" value="+41794492563" target="_blank" moz-do-not-send="true">+41 79 449 25 63</a> | </font> </font> </font> <a href="http://its.unibas.ch/" target="_blank" moz-do-not-send="true"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> http://its.unibas.ch </font> </font> </a><br> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> ITS-ServiceDesk: <a class="gmail-m_-6999624817399485079gmail-m_-1388566559526568039moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch" target="_blank" moz-do-not-send="true">support-its@unibas.ch</a> | <a href="tel:+41%2061%20267%2014%2011" value="+41612671411" target="_blank" moz-do-not-send="true">+41 61 267 14 11</a> </font> </font> </font> </font></p> <font color="#000000"> </font></div> </span></div> <br> ______________________________<wbr>_________________<br> Users mailing list<br> <a href="mailto:Users@ovirt.org" target="_blank" moz-do-not-send="true">Users@ovirt.org</a><br> <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br> <br> </blockquote> </div> <br> </div> </div> </blockquote> <br> <div class="gmail-m_-6999624817399485079moz-signature">-- <br> <p class="gmail-m_-6999624817399485079western" style="margin-bottom:0in;line-height:150%"> <font color="#000000"><font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> <b>Arsène Gschwind</b> </font> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> </font> </font> </font> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> </font> </font> <font face="Tahoma, serif"> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> <br> </font> </font> </font> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> Fa. Sapify AG im Auftrag der Universität Basel<br> IT Services<br> Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br> Tel. <a href="tel:+41%2079%20449%2025%2063" value="+41794492563" target="_blank" moz-do-not-send="true">+41 79 449 25 63</a> | </font> </font> </font> <a href="http://its.unibas.ch/" target="_blank" moz-do-not-send="true"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> http://its.unibas.ch </font> </font> </a><br> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size:8pt" size="1"> ITS-ServiceDesk: <a class="gmail-m_-6999624817399485079moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch" target="_blank" moz-do-not-send="true">support-its@unibas.ch</a> | <a href="tel:+41%2061%20267%2014%2011" value="+41612671411" target="_blank" moz-do-not-send="true">+41 61 267 14 11</a> </font> </font> </font> </font></p> <font color="#000000"> </font></div> </div> </div> </div> </blockquote> </div> <br> </div> </div> </blockquote> <br> <div class="moz-signature">-- <br> <p class="western" style="margin-bottom: 0in; line-height: 150%"> <font color="#000000"><font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> <b>Arsène Gschwind</b> </font> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> </font> </font> </font> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> </font> </font> <font face="Tahoma, serif"> </font> <font color="#000000"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> <br> </font> </font> </font> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> Fa. Sapify AG im Auftrag der Universität Basel<br> IT Services<br> Klingelbergstr. 70 | CH-4056 Basel | Switzerland<br> Tel. +41 79 449 25 63 | </font> </font> </font> <a href="http://its.unibas.ch/"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> http://its.unibas.ch </font> </font> </a><br> <font color="#7f7f7f"> <font face="Tahoma, serif"> <font style="font-size: 8pt" size="1"> ITS-ServiceDesk: <a class="moz-txt-link-abbreviated" href="mailto:support-its@unibas.ch">support-its@unibas.ch</a> | +41 61 267 14 11 </font> </font> </font> </font></p> <font color="#000000"> </font></div> </body> </html> --------------404D1197B599BE7464062905--
participants (3)
-
Arsène Gschwind -
Simone Tiraboschi -
Yedidyah Bar David