On Friday, April 13, 2018 6:48:31 PM EDT ~Stack~ wrote:
On 04/13/2018 07:16 AM, Alexander Wels wrote:
On Thursday, April 12, 2018 6:26:07 PM EDT ~Stack~ wrote:
Greetings,
So I did a over-confident-admin-makes-rookie-mistake. I changed a bunch of things all back-to-back and thus don't actually know what broke. :-D
The only two real "big" changes were: * Upgrade from 4.2.1 to 4.2.2 * change my ovirtmgmt network
The update I followed the upgrade procedures and I thought it all went pretty well. Because I am moving it from a testing into what I hope will be a more heavily used environment, I changed my ovirtmgmt network from 192.168.100.0/24 to 192.168.101.0/24 via the web-gui.
That was a touch tricker than just a change as I had to poke the management engine host to be reachable on both network for a while, then it just seemed OK.
What's happening is: * I can no longer migrate a vm from one host to the other. * If I try to do a "reinstall" it dies. * There is some serious network lag between my hosts on a 10Gb network. * I've got all kinds of python2.4 failures in my vdsm and mom logs.
Those are least the biggies.
So while I was planning on moving this to a more active use case, right now - it is all still my play ground. I would REALLY hate to lose the VM's but everything else can go and be rebuilt.
Given that I've somehow really broke this system pretty good, would it be more advisable to blow away and rebuild it ALL or can I simply delete the hypervisor hosts and rebuild them?
Thoughts?
Thanks! ~Stack~
As long as you don't destroy the data on your data domain you can rebuild the engine and hosts and then import the existing data domain without too many issues. I have destroyed my engine database many times, and I am still using the same VMs from the same data domain.
Here is what I do when I mess up my database to the point I have to make a new one:
1. Recreate the engine and database, so that I have basically have an empty engine with no hosts and VMs. 1.1 (Optional) make a new DC that is not default. and add a cluster. 2. Add my hosts (I only have 2 so that is quick and easy). 3. Add a throw away data domain (This is needed to get the DC up so I can import the existing data domain). 4. Import (NOT new, import) the existing data domain. 5. Do to Storage->Storage Domains->VM import and import the VMs I want. 6. Same for templates and disks if needed. 7. After you have imported the VMs/Templates/Disks you can detach and remove the throw away data domain and the one you imported becomes the master domain.
Note if you want to move VMs between your play ground and more serious system you can simply detach your data domain from the play ground, then attach it to the serious engine (so you have 2 engines, one play ground and one serious) and import which VMs you want. That way you won't run into issues with configuring networks and stuff like you experienced.
Thanks for that help. I did that and everything looks fantastic...except I can't migrate VM's. :-/
It just sits there and in the log files there is the below messages repeating. It's like it doesn't care for the fact that this was an imported domain or something.
Thoughts?
Thanks! ~Stack~
Don't know too much about the VDSM side of things. But obviously its looking for a storage domain it can't find anymore. You can try restarting VDSM (won't affect running VMs) and see if rescans the available storage domains and won't try to access it during the migration of the VMs. Other than that I don't know.
2018-04-13 16:58:59,920-0500 ERROR (monitor/232975a) [storage.Monitor] Setting up monitor for 232975ad-1771-4b6b-afda-958f7b745867 failed (monitor:329) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 326, in _setupLoop self._setupMonitor() File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 348, in _setupMonitor self._produceDomain() File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 158, in wrapper value = meth(self, *a, **kw) File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 366, in _produceDomain self.domain = sdCache.produce(self.sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110, in produce domain.getRealDomain() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 151, in _findDomain return findMethod(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 176, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'232975ad-1771-4b6b-afda-958f7b745867',) 2018-04-13 16:58:59,923-0500 ERROR (monitor/bc975a4) [storage.Monitor] Setting up monitor for bc975a4c-6c38-4248-b3f7-a26945f23693 failed (monitor:329) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 326, in _setupLoop self._setupMonitor() File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 348, in _setupMonitor self._produceDomain() File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 158, in wrapper value = meth(self, *a, **kw) File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 366, in _produceDomain self.domain = sdCache.produce(self.sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110, in produce domain.getRealDomain() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 151, in _findDomain return findMethod(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 176, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'bc975a4c-6c38-4248-b3f7-a26945f23693',)