Hi Giorgio, Can you please attach vdsm.log from both hosts, and engine.log? Try maybe moving both hosts to maintenance, "confirm host had been rebooted", and activate again. See if it helps.

StorageDomainDoesNotExist: Storage domain does not exist: (u'1810e5eb-9eb6-4797-ac50-8023a939f312',) What's the output of: lvs vdsClient -s 0 getStorageDomainsList If it exists in the list, please run: vdsClient -s 0 getStorageDomainInfo 1810e5eb-9eb6-4797-ac50-8023a939f312

Master data domain must be reachable in order for the DC to be up. Export domain shouldn't affect the dc status. Are you sure that you've created the export domain as an export domain, and not as a regular nfs?

----- Original Message ----- > From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > Sent: Tuesday, March 4, 2014 4:16:19 PM > Subject: Re: [Users] Data Center Non Responsive / Contending > > 2014-03-04 14:48 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > > StorageDomainDoesNotExist: Storage domain does not exist: > > (u'1810e5eb-9eb6-4797-ac50-8023a939f312',) > > > > What's the output of: > > lvs > > vdsClient -s 0 getStorageDomainsList > > > > If it exists in the list, please run: > > vdsClient -s 0 getStorageDomainInfo 1810e5eb-9eb6-4797-ac50-8023a939f312 > > > > I'm attaching a compressed archive to avoid mangling by googlemail client. > > Indeed the NFS storage with that id is not in the list of available > storage as it is brought up by a VM that has to be run in this very > same cluster. Obviously it isn't running at the moment. > > You find this in the DB: > > COPY storage_domain_static (id, storage, storage_name, > storage_domain_type, storage_type, storage_domain_format_type, > _create_date, _update_date, recoverable, last_time_used_as_master, > storage_description, storage_comment) FROM stdin; > ... > 1810e5eb-9eb6-4797-ac50-8023a939f312 > 11d4972d-f227-49ed-b997-f33cf4b2aa26 nfs02EXPORT 3 1 > 0 2014-02-28 18:11:23.17092+01 \N t 0 \N > \N > ... > > Also, disks for that VM are carved from the Master Data Domain that is > not available ATM. > > To say in other words: I thought that availability of an export domain > wasn't critical to switch on a Data Center. Am I wrong? > > Thanks, > Giorgio. >

Ok, and is the iscsi functional at the moment?

----- Original Message ----- > From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > Sent: Tuesday, March 4, 2014 4:35:07 PM > Subject: Re: [Users] Data Center Non Responsive / Contending > > 2014-03-04 15:23 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > > Master data domain must be reachable in order for the DC to be up. > > Export domain shouldn't affect the dc status. > > Are you sure that you've created the export domain as an export domain, and > > not as a regular nfs? > > > > Yes, I am. > > Don't know how to extract this info from DB, but in webadmin, in the > storage list, I have these info: > > Domain Name: nfs02EXPORT > Domain Type: Export > Storage Type: NFS > Format: V1 > Cross Data-Center Status: Inactive > Total Space: [N/A] > Free Space: [N/A] > > ATM my only "Data" Domain is based on iSCSI, no NFS. > > > > > > > ----- Original Message ----- > >> From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > >> To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > >> Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > >> Sent: Tuesday, March 4, 2014 4:16:19 PM > >> Subject: Re: [Users] Data Center Non Responsive / Contending > >> > >> 2014-03-04 14:48 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > >> > StorageDomainDoesNotExist: Storage domain does not exist: > >> > (u'1810e5eb-9eb6-4797-ac50-8023a939f312',) > >> > > >> > What's the output of: > >> > lvs > >> > vdsClient -s 0 getStorageDomainsList > >> > > >> > If it exists in the list, please run: > >> > vdsClient -s 0 getStorageDomainInfo 1810e5eb-9eb6-4797-ac50-8023a939f312 > >> > > >> > >> I'm attaching a compressed archive to avoid mangling by googlemail client. > >> > >> Indeed the NFS storage with that id is not in the list of available > >> storage as it is brought up by a VM that has to be run in this very > >> same cluster. Obviously it isn't running at the moment. > >> > >> You find this in the DB: > >> > >> COPY storage_domain_static (id, storage, storage_name, > >> storage_domain_type, storage_type, storage_domain_format_type, > >> _create_date, _update_date, recoverable, last_time_used_as_master, > >> storage_description, storage_comment) FROM stdin; > >> ... > >> 1810e5eb-9eb6-4797-ac50-8023a939f312 > >> 11d4972d-f227-49ed-b997-f33cf4b2aa26 nfs02EXPORT 3 1 > >> 0 2014-02-28 18:11:23.17092+01 \N t 0 \N > >> \N > >> ... > >> > >> Also, disks for that VM are carved from the Master Data Domain that is > >> not available ATM. > >> > >> To say in other words: I thought that availability of an export domain > >> wasn't critical to switch on a Data Center. Am I wrong? > >> > >> Thanks, > >> Giorgio. > >> >

From: "Liron Aravot" <laravot(a)redhat.com&gt; To: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; Sent: Tuesday, March 4, 2014 5:11:36 PM Subject: Re: [Users] Data Center Non Responsive / Contending ----- Original Message ----- > From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > Sent: Tuesday, March 4, 2014 5:06:13 PM > Subject: Re: [Users] Data Center Non Responsive / Contending > > 2014-03-04 15:38 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > > Ok, and is the iscsi functional at the moment? > > > > I think so. > For example I see in the DB that the id of my Master Data Domain , > dt02clu6070, is "a689cb30-743e-4261-bfd1-b8b194dc85db" then > > [root@vbox70 ~]# lvs a689cb30-743e-4261-bfd1-b8b194dc85db > LV VG > Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert > 4a1be3d8-ac7d-46cf-ae1c-ba154bc9a400 > a689cb30-743e-4261-bfd1-b8b194dc85db -wi------- 3,62g > 5c8bb733-4b0c-43a9-9471-0fde3d159fb2 > a689cb30-743e-4261-bfd1-b8b194dc85db -wi------- 11,00g > 7b617ab1-70c1-42ea-9303-ceffac1da72d > a689cb30-743e-4261-bfd1-b8b194dc85db -wi------- 3,88g > e4b86b91-80ec-4bba-8372-10522046ee6b > a689cb30-743e-4261-bfd1-b8b194dc85db -wi------- 9,00g > ids > a689cb30-743e-4261-bfd1-b8b194dc85db -wi-ao---- 128,00m > inbox > a689cb30-743e-4261-bfd1-b8b194dc85db -wi-a----- 128,00m > leases > a689cb30-743e-4261-bfd1-b8b194dc85db -wi-a----- 2,00g > master > a689cb30-743e-4261-bfd1-b8b194dc85db -wi-a----- 1,00g > metadata > a689cb30-743e-4261-bfd1-b8b194dc85db -wi-a----- 512,00m > outbox > a689cb30-743e-4261-bfd1-b8b194dc85db -wi-a----- 128,00m > > I can read from the LVs that have the LVM Available bit set: > > [root@vbox70 ~]# dd if=/dev/a689cb30-743e-4261-bfd1-b8b194dc85db/ids > bs=1M of=/dev/null > 128+0 records in > 128+0 records out > 134217728 bytes (134 MB) copied, 0,0323692 s, 4,1 GB/s > > [root@vbox70 ~]# dd if=/dev/a689cb30-743e-4261-bfd1-b8b194dc85db/ids > bs=1M |od -xc |head -20 > 0000000 2010 1221 0002 0003 0000 0000 0200 0000 > 020 ! 022 002 \0 003 \0 \0 \0 \0 \0 \0 002 \0 \0 > 0000020 0000 0000 0000 0000 0001 0000 0000 0000 > \0 \0 \0 \0 \0 \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 > 0000040 0001 0000 0000 0000 0007 0000 0000 0000 > 001 \0 \0 \0 \0 \0 \0 \0 \a \0 \0 \0 \0 \0 \0 \0 > 0000060 0000 0000 0000 0000 3661 3938 6263 3033 > \0 \0 \0 \0 \0 \0 \0 \0 a 6 8 9 c b 3 0 > 0000100 372d 3334 2d65 3234 3136 622d 6466 2d31 > - 7 4 3 e - 4 2 6 1 - b f d 1 - > 0000120 3862 3162 3439 6364 3538 6264 0000 0000 > b 8 b 1 9 4 d c 8 5 d b \0 \0 \0 \0 > 0000140 0000 0000 0000 0000 3638 3438 3966 3932 > \0 \0 \0 \0 \0 \0 \0 \0 8 6 8 4 f 9 2 9 > 0000160 612d 6237 2d66 3834 6564 622d 3830 2d35 > - a 7 b f - 4 8 d e - b 0 8 5 - > 0000200 6563 6330 6539 3537 6630 6364 762e 6f62 > c e 0 c 9 e 7 5 0 f d c . v b o > 0000220 3778 2e30 7270 0069 26de 0000 0000 0000 > x 7 0 . p r i \0 336 & \0 \0 \0 \0 \0 \0 > [root@vbox70 ~]# > > Obviously I can't read from LVs that aren't available: > > [root@vbox70 ~]# dd > if=/dev/a689cb30-743e-4261-bfd1-b8b194dc85db/4a1be3d8-ac7d-46cf-ae1c-ba154bc9a400 > bs=1M of=/dev/null > dd: apertura di > `/dev/a689cb30-743e-4261-bfd1-b8b194dc85db/4a1be3d8-ac7d-46cf-ae1c-ba154bc9a400': > No such file or directory > [root@vbox70 ~]# > > But those LV are the VM's disks and I suppose it's availability is > managed by oVirt > please see my previous mail on this thread, the issue seems to be with the connectivity to the nfs path, not the iscsi. 2014-03-04 13:15:41,167 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-27) [1141851d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to connect Host vbox70 to the Storage Domains nfs02EXPORT. > > > > ----- Original Message ----- > >> From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > >> To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > >> Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > >> Sent: Tuesday, March 4, 2014 4:35:07 PM > >> Subject: Re: [Users] Data Center Non Responsive / Contending > >> > >> 2014-03-04 15:23 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > >> > Master data domain must be reachable in order for the DC to be up. > >> > Export domain shouldn't affect the dc status. > >> > Are you sure that you've created the export domain as an export > >> > domain, > >> > and > >> > not as a regular nfs? > >> > > >> > >> Yes, I am. > >> > >> Don't know how to extract this info from DB, but in webadmin, in the > >> storage list, I have these info: > >> > >> Domain Name: nfs02EXPORT > >> Domain Type: Export > >> Storage Type: NFS > >> Format: V1 > >> Cross Data-Center Status: Inactive > >> Total Space: [N/A] > >> Free Space: [N/A] > >> > >> ATM my only "Data" Domain is based on iSCSI, no NFS. > >> > >> > >> > >> > >> > >> > ----- Original Message ----- > >> >> From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > >> >> To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > >> >> Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > >> >> Sent: Tuesday, March 4, 2014 4:16:19 PM > >> >> Subject: Re: [Users] Data Center Non Responsive / Contending > >> >> > >> >> 2014-03-04 14:48 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > >> >> > StorageDomainDoesNotExist: Storage domain does not exist: > >> >> > (u'1810e5eb-9eb6-4797-ac50-8023a939f312',) > >> >> > > >> >> > What's the output of: > >> >> > lvs > >> >> > vdsClient -s 0 getStorageDomainsList > >> >> > > >> >> > If it exists in the list, please run: > >> >> > vdsClient -s 0 getStorageDomainInfo > >> >> > 1810e5eb-9eb6-4797-ac50-8023a939f312 > >> >> > > >> >> > >> >> I'm attaching a compressed archive to avoid mangling by googlemail > >> >> client. > >> >> > >> >> Indeed the NFS storage with that id is not in the list of available > >> >> storage as it is brought up by a VM that has to be run in this very > >> >> same cluster. Obviously it isn't running at the moment. > >> >> > >> >> You find this in the DB: > >> >> > >> >> COPY storage_domain_static (id, storage, storage_name, > >> >> storage_domain_type, storage_type, storage_domain_format_type, > >> >> _create_date, _update_date, recoverable, last_time_used_as_master, > >> >> storage_description, storage_comment) FROM stdin; > >> >> ... > >> >> 1810e5eb-9eb6-4797-ac50-8023a939f312 > >> >> 11d4972d-f227-49ed-b997-f33cf4b2aa26 nfs02EXPORT 3 1 > >> >> 0 2014-02-28 18:11:23.17092+01 \N t 0 \N > >> >> \N > >> >> ... > >> >> > >> >> Also, disks for that VM are carved from the Master Data Domain that > >> >> is > >> >> not available ATM. > >> >> > >> >> To say in other words: I thought that availability of an export > >> >> domain > >> >> wasn't critical to switch on a Data Center. Am I wrong? > >> >> > >> >> Thanks, > >> >> Giorgio. > >> >> > >> > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

From: "Liron Aravot" <laravot(a)redhat.com&gt; To: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; Sent: Tuesday, March 4, 2014 5:03:44 PM Subject: Re: [Users] Data Center Non Responsive / Contending Hi Giorgio, Apperantly the issue is caused because there is no connectivity to the export domain and than we fail on spmStart - that's obviously a bug that shouldn't happen. can you open a bug for the issue? in the meanwhile, as it seems to still exist - seems to me like the way for solving it would be either to fix the connectivity issue between vdsm and the storage domain or to downgrade your vdsm version to before this issue was introduced.

6a519e95-62ef-445b-9a98-f05c81592c85::WARNING::2014-03-04 13:05:31,489::lvm::377::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "1810e5eb-9e b6-4797-ac50-8023a939f312" not found', ' Skipping volume group 1810e5eb-9eb6-4797-ac50-8023a939f312'] 6a519e95-62ef-445b-9a98-f05c81592c85::ERROR::2014-03-04 13:05:31,499::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 1810e5eb-9eb6-4797-ac50-8023a 939f312 not found Traceback (most recent call last): File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'1810e5eb-9eb6-4797-ac50-8023a939f312',) 6a519e95-62ef-445b-9a98-f05c81592c85::ERROR::2014-03-04 13:05:31,500::sp::329::Storage.StoragePool::(startSpm) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/sp.py", line 296, in startSpm self._updateDomainsRole() File "/usr/share/vdsm/storage/securable.py", line 75, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 205, in _updateDomainsRole domain = sdCache.produce(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 98, in produce domain.getRealDomain() File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce domain = self._findDomain(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) ----- Original Message ----- > From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > Sent: Tuesday, March 4, 2014 4:35:07 PM > Subject: Re: [Users] Data Center Non Responsive / Contending > > 2014-03-04 15:23 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > > Master data domain must be reachable in order for the DC to be up. > > Export domain shouldn't affect the dc status. > > Are you sure that you've created the export domain as an export domain, > > and > > not as a regular nfs? > > > > Yes, I am. > > Don't know how to extract this info from DB, but in webadmin, in the > storage list, I have these info: > > Domain Name: nfs02EXPORT > Domain Type: Export > Storage Type: NFS > Format: V1 > Cross Data-Center Status: Inactive > Total Space: [N/A] > Free Space: [N/A] > > ATM my only "Data" Domain is based on iSCSI, no NFS. > > > > > > > ----- Original Message ----- > >> From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > >> To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > >> Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > >> Sent: Tuesday, March 4, 2014 4:16:19 PM > >> Subject: Re: [Users] Data Center Non Responsive / Contending > >> > >> 2014-03-04 14:48 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > >> > StorageDomainDoesNotExist: Storage domain does not exist: > >> > (u'1810e5eb-9eb6-4797-ac50-8023a939f312',) > >> > > >> > What's the output of: > >> > lvs > >> > vdsClient -s 0 getStorageDomainsList > >> > > >> > If it exists in the list, please run: > >> > vdsClient -s 0 getStorageDomainInfo > >> > 1810e5eb-9eb6-4797-ac50-8023a939f312 > >> > > >> > >> I'm attaching a compressed archive to avoid mangling by googlemail > >> client. > >> > >> Indeed the NFS storage with that id is not in the list of available > >> storage as it is brought up by a VM that has to be run in this very > >> same cluster. Obviously it isn't running at the moment. > >> > >> You find this in the DB: > >> > >> COPY storage_domain_static (id, storage, storage_name, > >> storage_domain_type, storage_type, storage_domain_format_type, > >> _create_date, _update_date, recoverable, last_time_used_as_master, > >> storage_description, storage_comment) FROM stdin; > >> ... > >> 1810e5eb-9eb6-4797-ac50-8023a939f312 > >> 11d4972d-f227-49ed-b997-f33cf4b2aa26 nfs02EXPORT 3 1 > >> 0 2014-02-28 18:11:23.17092+01 \N t 0 \N > >> \N > >> ... > >> > >> Also, disks for that VM are carved from the Master Data Domain that is > >> not available ATM. > >> > >> To say in other words: I thought that availability of an export domain > >> wasn't critical to switch on a Data Center. Am I wrong? > >> > >> Thanks, > >> Giorgio. > >> > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

Hi Giorgio, Apperantly the issue is caused because there is no connectivity to the export domain and than we fail on spmStart - that's obviously a bug that shouldn't happen.

can you open a bug for the issue?

in the meanwhile, as it seems to still exist - seems to me like the way for solving it would be either to fix the connectivity issue between vdsm and the storage domain or to downgrade your vdsm version to before this issue was introduced.

6a519e95-62ef-445b-9a98-f05c81592c85::WARNING::2014-03-04 13:05:31,489::lvm::377::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "1810e5eb-9e b6-4797-ac50-8023a939f312" not found', ' Skipping volume group 1810e5eb-9eb6-4797-ac50-8023a939f312'] 6a519e95-62ef-445b-9a98-f05c81592c85::ERROR::2014-03-04 13:05:31,499::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 1810e5eb-9eb6-4797-ac50-8023a 939f312 not found Traceback (most recent call last): File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'1810e5eb-9eb6-4797-ac50-8023a939f312',) 6a519e95-62ef-445b-9a98-f05c81592c85::ERROR::2014-03-04 13:05:31,500::sp::329::Storage.StoragePool::(startSpm) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/sp.py", line 296, in startSpm self._updateDomainsRole() File "/usr/share/vdsm/storage/securable.py", line 75, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 205, in _updateDomainsRole domain = sdCache.produce(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 98, in produce domain.getRealDomain() File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce domain = self._findDomain(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) ----- Original Message ----- > From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > Sent: Tuesday, March 4, 2014 4:35:07 PM > Subject: Re: [Users] Data Center Non Responsive / Contending > > 2014-03-04 15:23 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > > Master data domain must be reachable in order for the DC to be up. > > Export domain shouldn't affect the dc status. > > Are you sure that you've created the export domain as an export domain, and > > not as a regular nfs? > > > > Yes, I am. > > Don't know how to extract this info from DB, but in webadmin, in the > storage list, I have these info: > > Domain Name: nfs02EXPORT > Domain Type: Export > Storage Type: NFS > Format: V1 > Cross Data-Center Status: Inactive > Total Space: [N/A] > Free Space: [N/A] > > ATM my only "Data" Domain is based on iSCSI, no NFS. > > > > > > > ----- Original Message ----- > >> From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > >> To: "Meital Bourvine" <mbourvin(a)redhat.com&gt; > >> Cc: &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt; > >> Sent: Tuesday, March 4, 2014 4:16:19 PM > >> Subject: Re: [Users] Data Center Non Responsive / Contending > >> > >> 2014-03-04 14:48 GMT+01:00 Meital Bourvine <mbourvin(a)redhat.com&gt;: > >> > StorageDomainDoesNotExist: Storage domain does not exist: > >> > (u'1810e5eb-9eb6-4797-ac50-8023a939f312',) > >> > > >> > What's the output of: > >> > lvs > >> > vdsClient -s 0 getStorageDomainsList > >> > > >> > If it exists in the list, please run: > >> > vdsClient -s 0 getStorageDomainInfo 1810e5eb-9eb6-4797-ac50-8023a939f312 > >> > > >> > >> I'm attaching a compressed archive to avoid mangling by googlemail client. > >> > >> Indeed the NFS storage with that id is not in the list of available > >> storage as it is brought up by a VM that has to be run in this very > >> same cluster. Obviously it isn't running at the moment. > >> > >> You find this in the DB: > >> > >> COPY storage_domain_static (id, storage, storage_name, > >> storage_domain_type, storage_type, storage_domain_format_type, > >> _create_date, _update_date, recoverable, last_time_used_as_master, > >> storage_description, storage_comment) FROM stdin; > >> ... > >> 1810e5eb-9eb6-4797-ac50-8023a939f312 > >> 11d4972d-f227-49ed-b997-f33cf4b2aa26 nfs02EXPORT 3 1 > >> 0 2014-02-28 18:11:23.17092+01 \N t 0 \N > >> \N > >> ... > >> > >> Also, disks for that VM are carved from the Master Data Domain that is > >> not available ATM. > >> > >> To say in other words: I thought that availability of an export domain > >> wasn't critical to switch on a Data Center. Am I wrong? > >> > >> Thanks, > >> Giorgio. > >> > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users >

----- Original Message ----- > From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > To: "Liron Aravot" <laravot(a)redhat.com&gt; > Cc: "Meital Bourvine" <mbourvin(a)redhat.com&gt;, &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt;, fsimonce(a)redhat.com > Sent: Tuesday, March 4, 2014 5:31:01 PM > Subject: Re: [Users] Data Center Non Responsive / Contending > > 2014-03-04 16:03 GMT+01:00 Liron Aravot <laravot(a)redhat.com&gt;: > > Hi Giorgio, > > Apperantly the issue is caused because there is no connectivity to the > > export domain and than we fail on spmStart - that's obviously a bug that > > shouldn't happen. > > Hi Liron, > we are reaching the same conclusion. > > > can you open a bug for the issue? > Surely I will > > > in the meanwhile, as it seems to still exist - seems to me like the way for > > solving it would be either to fix the connectivity issue between vdsm and > > the storage domain or to downgrade your vdsm version to before this issue > > was introduced. > > > I have some problems with your suggestion(s): > - I cannot fix the connectivity between vdsm and the storage domain > because, as I already said, it is exposed by a VM by this very same > DataCenter and if the DC doesn't goes up, the NFS server can't too. > - I don't understand what does it mean to downgrade the vdsm: to which > point in time? > > It seems I've put myself - again - in a situation of the "the egg or > the chicken" type, where the SD depends from THIS export domain but > the export domain isn't available if the DC isn't running. > > This export domain isn't that important to me. I can throw it away > without any problem. > > What if we edit the DB and remove any instances related to it? Any > adverse consequences? > Ok, please perform a full db backup before attempting the following: 1. right click on the the domain and choose "Destory" 2. move all hosts to maintenance 3. log in into the database and run the following sql command: update storage_pool where id = '{you id goes here}' set master_domain_version = master_domain_version + 1; 4. activate a host.

----- Original Message ----- > From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > To: "Liron Aravot" <laravot(a)redhat.com&gt; > Cc: "Meital Bourvine" <mbourvin(a)redhat.com&gt;, &quot;users(a)ovirt.org&quot; <Users(a)ovirt.org&gt;, fsimonce(a)redhat.com > Sent: Tuesday, March 4, 2014 6:10:27 PM > Subject: Re: [Users] Data Center Non Responsive / Contending > > 2014-03-04 16:37 GMT+01:00 Liron Aravot <laravot(a)redhat.com&gt;: > > > > > > ----- Original Message ----- > >> From: "Giorgio Bersano" <giorgio.bersano(a)gmail.com&gt; > >> To: "Liron Aravot" <laravot(a)redhat.com&gt; > >> Cc: "Meital Bourvine" <mbourvin(a)redhat.com&gt;, &quot;users(a)ovirt.org&quot; > >> <Users(a)ovirt.org&gt;, fsimonce(a)redhat.com > >> Sent: Tuesday, March 4, 2014 5:31:01 PM > >> Subject: Re: [Users] Data Center Non Responsive / Contending > >> > >> 2014-03-04 16:03 GMT+01:00 Liron Aravot <laravot(a)redhat.com&gt;: > >> > Hi Giorgio, > >> > Apperantly the issue is caused because there is no connectivity to the > >> > export domain and than we fail on spmStart - that's obviously a bug that > >> > shouldn't happen. > >> > >> Hi Liron, > >> we are reaching the same conclusion. > >> > >> > can you open a bug for the issue? > >> Surely I will > >> > >> > in the meanwhile, as it seems to still exist - seems to me like the way > >> > for > >> > solving it would be either to fix the connectivity issue between vdsm > >> > and > >> > the storage domain or to downgrade your vdsm version to before this > >> > issue > >> > was introduced. > >> > >> > >> I have some problems with your suggestion(s): > >> - I cannot fix the connectivity between vdsm and the storage domain > >> because, as I already said, it is exposed by a VM by this very same > >> DataCenter and if the DC doesn't goes up, the NFS server can't too. > >> - I don't understand what does it mean to downgrade the vdsm: to which > >> point in time? > >> > >> It seems I've put myself - again - in a situation of the "the egg or > >> the chicken" type, where the SD depends from THIS export domain but > >> the export domain isn't available if the DC isn't running. > >> > >> This export domain isn't that important to me. I can throw it away > >> without any problem. > >> > >> What if we edit the DB and remove any instances related to it? Any > >> adverse consequences? > >> > > > > Ok, please perform a full db backup before attempting the following: > > 1. right click on the the domain and choose "Destory" > > 2. move all hosts to maintenance > > 3. log in into the database and run the following sql command: > > update storage_pool where id = '{you id goes here}' set > > master_domain_version = master_domain_version + 1; > > 4. activate a host. > > Ok Liron, that did the trick! >

> Up and running again, even that VM supposed to be the server acting as > export domain. > > Now I've to run away as I'm late to a meeting but tomorrow I'll file a > bug regarding this. > > Thanks to you and Meital for your assistance, > Giorgio. Sure, happy that everything is fine! >

Giorgio, I've added the following patch to resolve the issue: http://gerrit.ovirt.org/#/c/25424/ Have you opened the bug for it? if so, please provide me the bug number so i could assign the patch to it. Thanks.

Would you mind sharing the link to it? I didn't find it. Thanks!