[ovirt-users] New SAN Storage Domain: Cannot deactivate Logical Volume

Thu Apr 6 18:55:56 UTC 2017

Actually I was seeing something similar yesterday and I have seen very
sporadic failures in this area in the past.  It seems that in some cases
the VG is not properly set up before we try to use it (perhaps a race with
devicemapper).  I wonder if adding some sleeps in
BlockStorageDomain.create() would mask the issue.  If so, we need to figure
out where the race is happening and might need to add some polling to make
sure the underlying devices are ready.

On Thu, Apr 6, 2017 at 2:35 PM, Liron Aravot <laravot at redhat.com> wrote:

> Thanks Alexey,
> Nir/Adam/Raz - any initial thoughts about that? have you encountered that
> issue before?
>
> Regards,
> Liron
>
> On Wed, Apr 5, 2017 at 12:21 PM, Николаев Алексей <
> alexeynikolaev.post at yandex.ru> wrote:
>
>> Hi Liron,
>>
>> Well, this error appear on freshly installed oVirt hosted engine 4.1.1.6-1
>> with oVirt Node NG 4.1.
>> There is no engine major version upgrade.
>> Node NG was updated from 4.1.0 to 4.1.1 by the ovirt-node-ng-image* pkgs.
>>
>> vdsm version is:
>>
>> Installed Packages
>> Name        : vdsm
>> Arch        : x86_64
>> Version     : 4.19.10
>> Release     : 1.el7.centos
>> ------------------------------------------------------
>>
>> I also install oVirt 3.6 hosted engine on another server (CentOS 7.2
>> minimal, not updated to 7.3) and add the some ISCSI target as the data
>> domain. This work OK.
>> After I update engine from 3.6 to 4.1. ISCSI data domain is OK after
>> update. Another ISCSI targets was added as data domain without errors.
>>
>> vdsm version is:
>>
>> Installed Packages
>> Name        : vdsm
>> Arch        : noarch
>> Version     : 4.17.32
>> Release     : 0.el7.centos
>>
>> 04.04.2017, 16:48, "Liron Aravot" <laravot at redhat.com>:
>>
>> Hi Alexey,
>>
>> Looking in your logs - it indeed look weird, the vg is created seconds
>> later we fail to get it when attempting to create the domain.
>>
>> Did you upgrade/downgraded only the engine version or the vdsm version as
>> well?
>>
>> 2017-04-03 11:16:39,595+0300 INFO  (jsonrpc/4) [dispatcher] Run and
>> protect: createVG(vgname=u'3050e121-83b5-474e-943f-6dc52af38a17',
>> devlist=[u'3600140505d11d22e61f46a6ba4a
>> 7bdc7'], force=False, options=None) (logUtils:51)
>>
>>
>> output error', '  /dev/mapper/36001405b7dd7b800e7049ecbf6830637: read
>> failed after 0 of 4096 at 4294967230464: Input/output error', '
>>  /dev/mapper/36001405b7dd7b800e7049ecbf
>> 6830637: read failed after 0 of 4096 at 4294967287808: Input/output
>> error', '  WARNING: Error counts reached a limit of 3. Device /dev/mapper/
>> 36001405b7dd7b800e7049ecbf68306
>> 37 was disabled', '  Failed to find physical volume "/dev/mapper/
>> 3600140505d11d22e61f46a6ba4a7bdc7".'] (lvm:323)
>> 2017-04-03 11:16:38,832+0300 WARN  (jsonrpc/5) [storage.HSM] getPV
>> failed for guid: 3600140505d11d22e61f46a6ba4a7bdc7 (hsm:1973)
>> Traceback (most recent call last):
>>   File "/usr/share/vdsm/storage/hsm.py", line 1970, in _getDeviceList
>>     pv = lvm.getPV(guid)
>>   File "/usr/share/vdsm/storage/lvm.py", line 853, in getPV
>>     raise se.((pvName,))
>>
>>
>> On Tue, Apr 4, 2017 at 3:33 PM, Николаев Алексей <
>> alexeynikolaev.post at yandex.ru> wrote:
>>
>> The last I was update oVirt oVirt 3.6.7.5-1 to oVirt 4.1.1.6-1. Main
>> ISCSI datadomain is OK. Another one ISCSI datadomain was added
>> successfully.
>>
>> The problem are only with freshly installed oVirt 4.1.1.6-1.
>>
>> Added logs after update ovirt and add new ISCSI datadomains.
>>
>> 03.04.2017, 17:26, "Николаев Алексей" <alexeynikolaev.post at yandex.ru>:
>>
>> Also I test again ISCSI TARGET with oVirt 3.6.7.5-1 and CentOS 7.2.1511
>> host. ISCSI data domain was added succesfully. "the truth is out there".
>> Add logs.
>>
>> 03.04.2017, 11:24, "Николаев Алексей" <alexeynikolaev.post at yandex.ru>:
>>
>> Add logs. Thx!
>>
>>
>> 02.04.2017, 16:19, "Liron Aravot" <laravot at redhat.com>:
>>
>> Hi Alexey,
>> can you please attach the engine/vdsm logs?
>>
>> thanks.
>>
>> On Fri, Mar 31, 2017 at 11:08 AM, Николаев Алексей <
>> alexeynikolaev.post at yandex.ru> wrote:
>>
>> Hi, community!
>>
>> I'm try using ISCSI DATA DOMAIN with oVirt 4.1.1.6-1 and oVirt Node 4.1.1.
>> But get this error while adding data domain.
>>
>> Error while executing action New SAN Storage Domain: Cannot deactivate
>> Logical Volume
>>
>>
>> 2017-03-31 10:47:45,099+03 ERROR [org.ovirt.engine.core.vdsbrok
>> er.vdsbroker.CreateStorageDomainVDSCommand] (default task-41) [717af17f]
>> Command 'CreateStorageDomainVDSCommand(HostName = node169-07.,
>> CreateStorageDomainVDSCommandParameters:{runAsync='true',
>> hostId='ca177a14-56ef-4736-a0a9-ab9dc2a8eb90',
>> storageDomain='StorageDomainStatic:{name='data1', id='56038842-2
>> fbe-4ada-96f1-c4b4e66fd0b7'}', args='sUG5rL-zqvF-DMox-uMK3-Xl91-ZiYw-k27C3B'})'
>> execution failed: VDSGenericException: VDSErrorException: Failed in
>> vdscommand to CreateStorageDomainVDS, error = Cannot deactivate Logical
>> Volume: ('General Storage Exception: (\'5 [] [\\\'  /dev/mapper/36001405
>> b7dd7b800e7049ecbf6830637: read failed after 0 of 4096 at 0:
>> Input/output error\\\', \\\'  /dev/mapper/36001405b7dd7b800e7049ecbf
>> 6830637: read failed after 0 of 4096 at 4294967230464: Input/output
>> error\\\', \\\'  /dev/mapper/36001405b7dd7b800e7049ecbf6830637: read
>> failed after 0 of 4096 at 4294967287808: Input/output error\\\', \\\'
>> WARNING: Error counts reached a limit of 3. Device /dev/mapper/36001405
>> b7dd7b800e7049ecbf6830637 was disabled\\\', \\\'  WARNING: Error counts
>> reached a limit of 3. Device /dev/mapper/36001405b7dd7b800e7049ecbf
>> 6830637 was disabled\\\', \\\'  Volume group "56038842-2
>> fbe-4ada-96f1-c4b4e66fd0b7" not found\\\', \\\'  Cannot process volume
>> group 56038842-2fbe-4ada-96f1-c4b4e66fd0b7\\\']\\n56038842-2fbe-
>> 4ada-96f1-c4b4e66fd0b7/[\\\'master\\\']\',)',)
>>
>>
>> Previously, this ISCSI DATA DOMAIN works well with oVirt 3.6 and CentOS
>> 7.2 host.
>> How I can debug this error?
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>> ,
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>> ,
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>

-- 
Adam Litke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170406/d7199956/attachment.html>