1:08 p.m.
On Wed, Feb 13, 2019 at 8:06 PM Jayme <jaymef(a)gmail.com> wrote:
I might be hitting this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1666795
Yes, you definitively are.
Fixing files ownership on file system side is a valid workaround.
On Wed, Feb 13, 2019 at 1:35 PM Jayme <jaymef(a)gmail.com> wrote:
> This may be happening because I changed cluster compatibility to 4.3 then
> immediately after changed data center compatibility to 4.3 (before
> restarting VMs after cluster compatibility change). If this is the case I
> can't fix by downgrading the data center compatibility to 4.2 as it won't
> allow me to do so. What can I do to fix this, any VM I restart will break
> (I am leaving the others running for now, but there are some down that I
> can't start).
>
> Full error from VDSM:
>
> 2019-02-13 13:30:55,465-0400 ERROR (vm/d070ce80)
> [storage.TaskManager.Task] (Task='d5c8e50a-0a6f-4fe7-be79-fd322b273a1e')
> Unexpected error (task:875)
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
> in _run
> return fn(*args, **kargs)
> File "<string>", line 2, in prepareImage
> File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in
> method
> ret = func(*args, **kwargs)
> File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3198,
> in prepareImage
> legality = dom.produceVolume(imgUUID, volUUID).getLegality()
> File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 818,
> in produceVolume
> volUUID)
> File "/usr/lib/python2.7/site-packages/vdsm/storage/glusterVolume.py",
> line 45, in __init__
> volUUID)
> File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
> 800, in __init__
> self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID,
> volUUID)
> File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py",
> line 71, in __init__
> volUUID)
> File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
> 86, in __init__
> self.validate()
> File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
> 112, in validate
> self.validateVolumePath()
> File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py",
> line 131, in validateVolumePath
> raise se.VolumeDoesNotExist(self.volUUID)
> VolumeDoesNotExist: Volume does not exist:
> (u'2d6d5f87-ccb0-48ce-b3ac-84495bd12d32',)
> 2019-02-13 13:30:55,468-0400 ERROR (vm/d070ce80) [storage.Dispatcher]
> FINISH prepareImage error=Volume does not exist:
> (u'2d6d5f87-ccb0-48ce-b3ac-84495bd12d32',) (dispatcher:81)
> 2019-02-13 13:30:55,469-0400 ERROR (vm/d070ce80) [virt.vm]
> (vmId='d070ce80-e0bc-489d-8ee0-47d5926d5ae2') The vm start process failed
> (vm:937)
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 866, in
> _startUnderlyingVm
> self._run()
> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2749, in
> _run
> self._devices = self._make_devices()
> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2589, in
> _make_devices
> disk_objs = self._perform_host_local_adjustment()
> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2662, in
> _perform_host_local_adjustment
> self._preparePathsForDrives(disk_params)
> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1011, in
> _preparePathsForDrives
> drive['path'] = self.cif.prepareVolumePath(drive, self.id)
> File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 415, in
> prepareVolumePath
> raise vm.VolumeError(drive)
> VolumeError: Bad volume specification {'address': {'function':
'0x0',
> 'bus': '0x00', 'domain': '0x0000', 'type':
'pci', 'slot': '0x06'},
> 'serial': 'd81a6826-dc46-44db-8de7-405d30e44d57', 'index': 0,
'iface':
> 'virtio', 'apparentsize': '64293699584',
'specParams': {}, 'cache': 'none',
> 'imageID': 'd81a6826-dc46-44db-8de7-405d30e44d57',
'truesize':
> '64293814272', 'type': 'disk', 'domainID':
> '1f2e9989-9ab3-43d5-971d-568b8feca918', 'reqsize': '0',
'format': 'cow',
> 'poolID': 'a45e442e-9989-11e8-b0e4-00163e4bf18a', 'device':
'disk', 'path':
>
'/rhev/data-center/a45e442e-9989-11e8-b0e4-00163e4bf18a/1f2e9989-9ab3-43d5-971d-568b8feca918/images/d81a6826-dc46-44db-8de7-405d30e44d57/2d6d5f87-ccb0-48ce-b3ac-84495bd12d32',
> 'propagateErrors': 'off', 'name': 'vda',
'bootOrder': '1', 'volumeID':
> '2d6d5f87-ccb0-48ce-b3ac-84495bd12d32', 'diskType': 'file',
'alias':
> 'ua-d81a6826-dc46-44db-8de7-405d30e44d57', 'discard': False}
>
> On Wed, Feb 13, 2019 at 1:19 PM Jayme <jaymef(a)gmail.com> wrote:
>
>> I may have made matters worse. So I changed to 4.3 compatible cluster
>> then 4.3 compatible data center. All VMs were marked as requiring a
>> reboot. I restarted a couple of them and none of them will start up, they
>> are saying "bad volume specification". The ones running that I did not
yet
>> restart are still running ok. I need to figure out why the VMs aren't
>> restarting.
>>
>> Here is an example from vdsm.log
>>
>> olumeError: Bad volume specification {'address': {'function':
'0x0',
>> 'bus': '0x00', 'domain': '0x0000',
'type': 'pci', 'slot': '0x06'},
>> 'serial': 'd81a6826-dc46-44db-8de7-405d30e44d57',
'index': 0, 'iface':
>> 'virtio', 'apparentsize': '64293699584',
'specParams': {}, 'cache': 'none',
>> 'imageID': 'd81a6826-dc46-44db-8de7-405d30e44d57',
'truesize':
>> '64293814272', 'type': 'disk', 'domainID':
>> '1f2e9989-9ab3-43d5-971d-568b8feca918', 'reqsize': '0',
'format': 'cow',
>> 'poolID': 'a45e442e-9989-11e8-b0e4-00163e4bf18a',
'device': 'disk', 'path':
>>
'/rhev/data-center/a45e442e-9989-11e8-b0e4-00163e4bf18a/1f2e9989-9ab3-43d5-971d-568b8feca918/images/d81a6826-dc46-44db-8de7-405d30e44d57/2d6d5f87-ccb0-48ce-b3ac-84495bd12d32',
>> 'propagateErrors': 'off', 'name': 'vda',
'bootOrder': '1', 'volumeID':
>> '2d6d5f87-ccb0-48ce-b3ac-84495bd12d32', 'diskType':
'file', 'alias':
>> 'ua-d81a6826-dc46-44db-8de7-405d30e44d57', 'discard': False}
>>
>> On Wed, Feb 13, 2019 at 1:01 PM Jayme <jaymef(a)gmail.com> wrote:
>>
>>> I think I just figured out what I was doing wrong. On edit cluster
>>> screen I was changing both the CPU type and cluster level 4.3. I tried it
>>> again by switching to the new CPU type first (leaving cluster on 4.2) then
>>> saving, then going back in and switching compat level to 4.3. It appears
>>> that you need to do this in two steps for it to work.
>>>
>>>
>>>
>>> On Wed, Feb 13, 2019 at 12:57 PM Jayme <jaymef(a)gmail.com> wrote:
>>>
>>>> Hmm interesting, I wonder how you were able to switch from SandyBridge
>>>> IBRS to SandyBridge IBRS SSBD. I just attempted the same in both
regular
>>>> mode and in global maintenance mode and it won't allow me to, it says
that
>>>> all hosts have to be in maintenance mode (screenshots attached). Are
you
>>>> also running HCI/Gluster setup?
>>>>
>>>>
>>>>
>>>> On Wed, Feb 13, 2019 at 12:44 PM Ron Jerome <ronjero(a)gmail.com>
wrote:
>>>>
>>>>> > Environment setup:
>>>>> >
>>>>> > 3 Host HCI GlusterFS setup. Identical hosts, Dell R720s w/
Intel
>>>>> E5-2690
>>>>> > CPUs
>>>>> >
>>>>> > 1 default data center (4.2 compat)
>>>>> > 1 default cluster (4.2 compat)
>>>>> >
>>>>> > Situation: I recently upgraded my three node HCI cluster from
Ovirt
>>>>> 4.2 to
>>>>> > 4.3. I did so by first updating the engine to 4.3 then
upgrading
>>>>> each
>>>>> > ovirt-node host to 4.3 and rebooting.
>>>>> >
>>>>> > Currently engine and all hosts are running 4.3 and all is
working
>>>>> fine.
>>>>> >
>>>>> > To complete the upgrade I need to update cluster compatibility
to
>>>>> 4.3 and
>>>>> > then data centre to 4.3. This is where I am stuck.
>>>>> >
>>>>> > The CPU type on cluster is "Intel SandyBridge IBRS
Family". This
>>>>> option is
>>>>> > no longer available if I select 4.3 compatibility. Any other
>>>>> option chosen
>>>>> > such as SandyBridge IBRS SSBD will not allow me to switch to 4.3
as
>>>>> all
>>>>> > hosts must be in maintenance mode (which is not possible w/
self
>>>>> hosted
>>>>> > engine).
>>>>> >
>>>>> > I saw another post about this where someone else followed steps
to
>>>>> create a
>>>>> > second cluster on 4.3 with new CPU type then move one host to
it,
>>>>> start
>>>>> > engine on it then perform other steps to eventually get to 4.3
>>>>> > compatibility.
>>>>> >
>>>>>
>>>>> I have the exact same hardware configuration and was able to change
>>>>> to "SandyBridge IBRS SSBD" without creating a new cluster.
How I made that
>>>>> happen, I'm not so sure, but the cluster may have been in
"Global
>>>>> Maintenance" mode when I changed it.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list -- users(a)ovirt.org
>>>>> To unsubscribe send an email to users-leave(a)ovirt.org
>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>>> oVirt Code of Conduct:
>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>> List Archives:
>>>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/5B3TAXKO7IB...
>>>>>
>>>> _______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/PK7IR27DGLZ...