1:20 a.m.
On 10/13/2020 5:01 PM, Michael Thomas wrote:
After getting past the proxy issue, I was finally able to run the
engine-setup --reconfigure-optional-components. The new
ManagedBlockStorage storage domain exists, and I was able to create a
disk. However, I am unable to attach the disk to a running VM.
The engine.log shows the following, with a reference to a possible
cinderlib error ("cinderlib execution failed"):
2020-10-13 15:15:23,508-05 INFO
[org.ovirt.engine.core.bll.UpdateVmCommand] (default task-13)
[c73386d0-a713-4c37-bc9b-e7c4f9083f78] Lock Acquired to object
'EngineLock:{exclusiveLocks='[grafana=VM_NAME]',
sharedLocks='[5676d441-660e-4d9f-a586-e53ff0ea054b=VM]'}'
2020-10-13 15:15:23,522-05 INFO
[org.ovirt.engine.core.bll.UpdateVmCommand] (default task-13)
[c73386d0-a713-4c37-bc9b-e7c4f9083f78] Running command:
UpdateVmCommand internal: false. Entities affected : ID:
5676d441-660e-4d9f-a586-e53ff0ea054b Type: VMAction group
EDIT_VM_PROPERTIES with role type USER
2020-10-13 15:15:23,536-05 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-13) [c73386d0-a713-4c37-bc9b-e7c4f9083f78] EVENT_ID:
USER_UPDATE_VM(35), VM grafana configuration was updated by
michael.thomas@internal-authz.
2020-10-13 15:15:23,539-05 INFO
[org.ovirt.engine.core.bll.UpdateVmCommand] (default task-13)
[c73386d0-a713-4c37-bc9b-e7c4f9083f78] Lock freed to object
'EngineLock:{exclusiveLocks='[grafana=VM_NAME]',
sharedLocks='[5676d441-660e-4d9f-a586-e53ff0ea054b=VM]'}'
2020-10-13 15:15:24,129-05 INFO
[org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand]
(default task-13) [f8829338-b040-46d0-a838-3cf28869637c] Lock Acquired
to object
'EngineLock:{exclusiveLocks='[5419640e-445f-4b3f-a29d-b316ad031b7a=DISK]',
sharedLocks=''}'
2020-10-13 15:15:24,147-05 INFO
[org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand]
(default task-13) [f8829338-b040-46d0-a838-3cf28869637c] Running
command: AttachDiskToVmCommand internal: false. Entities affected :
ID: 5676d441-660e-4d9f-a586-e53ff0ea054b Type: VMAction group
CONFIGURE_VM_STORAGE with role type USER, ID:
5419640e-445f-4b3f-a29d-b316ad031b7a Type: DiskAction group
ATTACH_DISK with role type USER
2020-10-13 15:15:24,152-05 INFO
[org.ovirt.engine.core.bll.storage.disk.managedblock.ConnectManagedBlockStorageDeviceCommand]
(default task-13) [7cb262cc] Running command:
ConnectManagedBlockStorageDeviceCommand internal: true.
2020-10-13 15:15:26,006-05 ERROR
[org.ovirt.engine.core.common.utils.cinderlib.CinderlibExecutor]
(default task-13) [7cb262cc] cinderlib execution failed:
2020-10-13 15:15:26,011-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand]
(default task-13) [7cb262cc] START, HotPlugDiskVDSCommand(HostName =
ovirt4-mgmt.ldas.ligo-la.caltech.edu,
HotPlugDiskVDSParameters:{hostId='61da4cdf-638b-4cbd-9921-5be820998d31',
vmId='5676d441-660e-4d9f-a586-e53ff0ea054b',
diskId='5419640e-445f-4b3f-a29d-b316ad031b7a'}), log id: 660ebc9e
2020-10-13 15:15:26,012-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand]
(default task-13) [7cb262cc] Failed in 'HotPlugDiskVDS' method, for
vds: 'ovirt4-mgmt.ldas.ligo-la.caltech.edu'; host:
'ovirt4-mgmt.ldas.ligo-la.caltech.edu': null
2020-10-13 15:15:26,012-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand]
(default task-13) [7cb262cc] Command 'HotPlugDiskVDSCommand(HostName =
ovirt4-mgmt.ldas.ligo-la.caltech.edu,
HotPlugDiskVDSParameters:{hostId='61da4cdf-638b-4cbd-9921-5be820998d31',
vmId='5676d441-660e-4d9f-a586-e53ff0ea054b',
diskId='5419640e-445f-4b3f-a29d-b316ad031b7a'})' execution failed: null
2020-10-13 15:15:26,012-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand]
(default task-13) [7cb262cc] FINISH, HotPlugDiskVDSCommand, return: ,
log id: 660ebc9e
2020-10-13 15:15:26,012-05 ERROR
[org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand]
(default task-13) [7cb262cc] Command
'org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand' failed:
EngineException: java.lang.NullPointerException (Failed with error
ENGINE and code 5001)
2020-10-13 15:15:26,013-05 ERROR
[org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand]
(default task-13) [7cb262cc] Transaction rolled-back for command
'org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand'.
2020-10-13 15:15:26,021-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-13) [7cb262cc] EVENT_ID:
USER_FAILED_ATTACH_DISK_TO_VM(2,017), Failed to attach Disk
testvm_disk to VM grafana (User: michael.thomas@internal-authz).
2020-10-13 15:15:26,021-05 INFO
[org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand]
(default task-13) [7cb262cc] Lock freed to object
'EngineLock:{exclusiveLocks='[5419640e-445f-4b3f-a29d-b316ad031b7a=DISK]',
sharedLocks=''}'
The /var/log/cinder/ directory on the ovirt node is empty, and doesn't
exist on the engine itself.
To verify that it's not a cephx permission issue, I tried accessing
the block storage from both the engine and the ovirt node using the
credentials I set up in the ManagedBlockStorage setup page:
[root@ovirt4]# rbd --id ovirt ls rbd.ovirt.data
volume-5419640e-445f-4b3f-a29d-b316ad031b7a
[root@ovirt4]# rbd --id ovirt info
rbd.ovirt.data/volume-5419640e-445f-4b3f-a29d-b316ad031b7a
rbd image 'volume-5419640e-445f-4b3f-a29d-b316ad031b7a':
size 100 GiB in 25600 objects
order 22 (4 MiB objects)
snapshot_count: 0
id: 68a7cd6aeb3924
block_name_prefix: rbd_data.68a7cd6aeb3924
format: 2
features: layering, exclusive-lock, object-map, fast-diff,
deep-flatten
Remove the object-map, fast-diff and deep-flatten options from the rbd.
You need to be able to kernel map the rbd. Only one of the major
deficiencies of the cinder/cinderlib method...
> op_features:
> flags:
> create_timestamp: Tue Oct 13 06:53:55 2020
> access_timestamp: Tue Oct 13 06:53:55 2020
> modify_timestamp: Tue Oct 13 06:53:55 2020
>
> Where else can I look to see where it's failing?
>
> --Mike
>
> On 9/30/20 2:19 AM, Benny Zlotnik wrote:
>> When you ran `engine-setup` did you enable cinderlib preview (it will
>> not be enabled by default)?
>> It should handle the creation of the database automatically, if you
>> didn't you can enable it by running:
>> `engine-setup --reconfigure-optional-components`
>>
>>
>> On Wed, Sep 30, 2020 at 1:58 AM Michael Thomas <wart(a)caltech.edu> wrote:
>>>
>>> Hi Benny,
>>>
>>> Thanks for the confirmation. I've installed openstack-ussuri and ceph
>>> Octopus. Then I tried using these instructions, as well as the deep
>>> dive that Eyal has posted at
>>> https://www.youtube.com/watch?v=F3JttBkjsX8.
>>>
>>> I've done this a couple of times, and each time the engine fails when I
>>> try to add the new managed block storage domain. The error on the
>>> screen indicates that it can't connect to the cinder database. The
>>> error in the engine log is:
>>>
>>> 2020-09-29 17:02:11,859-05 WARN
>>>
[org.ovirt.engine.core.bll.storage.domain.AddManagedBlockStorageDomainCommand]
>>>
>>> (default task-2) [d519088c-7956-4078-b5cf-156e5b3f1e59] Validation of
>>> action 'AddManagedBlockStorageDomain' failed for user
>>> admin@internal-authz. Reasons:
>>>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ADD,ACTION_TYPE_FAILED_CINDERLIB_DATA_BASE_REQUIRED,ACTION_TYPE_FAILED_CINDERLIB_DATA_BASE_REQUIRED
>>>
>>>
>>> I had created the db on the engine with this command:
>>>
>>> su - postgres -c "psql -d template1 -c \"create database cinder
owner
>>> engine template template0 encoding 'UTF8' lc_collate
'en_US.UTF-8'
>>> lc_ctype 'en_US.UTF-8';\""
>>>
>>> ...and added the following to the end of
>>> /var/lib/pgsql/data/pg_hba.conf:
>>>
>>> host cinder engine ::0/0 md5
>>> host cinder engine 0.0.0.0/0 md5
>>>
>>> Is there anywhere else I should look to find out what may have gone
>>> wrong?
>>>
>>> --Mike
>>>
>>> On 9/29/20 3:34 PM, Benny Zlotnik wrote:
>>>> The feature is currently in tech preview, but it's being worked on.
>>>> The feature page is outdated, but I believe this is what most users
>>>> in the mailing list were using. We held off on updating it because the
>>>> installation instructions have been a moving target, but it is more
>>>> stable now and I will update it soon.
>>>>
>>>> Specifically speaking, the openstack version should be updated to
>>>> train (it is likely ussuri works fine too, but I haven't tried it)
and
>>>> cinderlib has an RPM now (python3-cinderlib)[1], so it can be
>>>> installed instead of using pip, same goes for os-brick. The rest of
>>>> the information is valid.
>>>>
>>>>
>>>> [1]
>>>>
http://mirror.centos.org/centos/8/cloud/x86_64/openstack-ussuri/Packages/p/
>>>>
>>>> On Tue, Sep 29, 2020 at 10:37 PM Michael Thomas <wart(a)caltech.edu>
>>>> wrote:
>>>>>
>>>>> I'm looking for the latest documentation for setting up a Managed
>>>>> Block
>>>>> Device storage domain so that I can move some of my VM images to
>>>>> ceph rbd.
>>>>>
>>>>> I found this:
>>>>>
>>>>>
https://ovirt.org/develop/release-management/features/storage/cinderlib-i...
>>>>>
>>>>>
>>>>> ...but it has a big note at the top that it is "...not user
>>>>> documentation and should not be treated as such."
>>>>>
>>>>> The oVirt administration guide[1] does not talk about managed
>>>>> block devices.
>>>>>
>>>>> I've found a few mailing list threads that discuss people setting
>>>>> up a
>>>>> Managed Block Device with ceph, but didn't see any links to
>>>>> documentation steps that folks were following.
>>>>>
>>>>> Is the Managed Block Storage domain a supported feature in oVirt
>>>>> 4.4.2,
>>>>> and if so, where is the documentation for using it?
>>>>>
>>>>> --Mike
>>>>> [1]ovirt.org/documentation/administration_guide/
>>>>> _______________________________________________
>>>>> Users mailing list -- users(a)ovirt.org
>>>>> To unsubscribe send an email to users-leave(a)ovirt.org
>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>> oVirt Code of Conduct:
>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>> List Archives:
>>>>>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KHCLXVOCELH...
>>>>
>>>
>>
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/GZX6634QCLB...