[ovirt-devel] [VDSM] All tests using directio fail on CI

[Nir Soffer]

On Wed, Sep 28, 2016 at 10:31 PM, Barak Korren <bkorren at redhat.com> wrote:
> The CI setup did not change recently.

Great

> All standard-CI jobs run inside mock (chroot) which is stored on top
> of a regular FS, so they should not be affected by the slave OS at all
> as far as FS settings go.
>
> But perhaps some slave-OS/mock-OS combination is acting strangely, so
> could you be more specific and point to particular job runs that fail?

This jobs failed, but it was deleted (I get 404 now):
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/2530/

Then I triggered the tests and they passed:
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/2548/

Thanks
Nir

>
> On 28 September 2016 at 22:00, Nir Soffer <nsoffer at redhat.com> wrote:
>> Hi all,
>>
>> It seems that the CI setup has changed, and /var/tmp is using now tempfs.
>>
>> This is not compatible with vdsm tests, assuming that /var/tmp is a real file
>> system. This is the reason we do not use /tmp.
>>
>> We have lot of storage tests using directio, and directio cannot work on
>> tempfs.
>>
>> Please check the slaves and make sure /var/tmp is using file system supporting
>> directio.
>>
>> See example failure bellow.
>>
>> Nir
>>
>> ----
>>
>> 12:33:20 ======================================================================
>> 12:33:20 ERROR: test_create_fail_creating_lease
>> (storage_volume_artifacts_test.BlockVolumeArtifactsTests)
>> 12:33:20 ----------------------------------------------------------------------
>> 12:33:20 Traceback (most recent call last):
>> 12:33:20   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/storage_volume_artifacts_test.py",
>> line 485, in test_create_fail_creating_lease
>> 12:33:20     *BASE_PARAMS[sc.RAW_FORMAT])
>> 12:33:20   File "/usr/lib64/python2.7/unittest/case.py", line 513, in
>> assertRaises
>> 12:33:20     callableObj(*args, **kwargs)
>> 12:33:20   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/sdm/volume_artifacts.py",
>> line 391, in create
>> 12:33:20     desc, parent)
>> 12:33:20   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/sdm/volume_artifacts.py",
>> line 482, in _create_metadata
>> 12:33:20     sc.LEGAL_VOL)
>> 12:33:20   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/volume.py",
>> line 427, in newMetadata
>> 12:33:20     cls.createMetadata(metaId, meta.legacy_info())
>> 12:33:20   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/volume.py",
>> line 420, in createMetadata
>> 12:33:20     cls._putMetadata(metaId, meta)
>> 12:33:20   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/blockVolume.py",
>> line 242, in _putMetadata
>> 12:33:20     f.write(data)
>> 12:33:20   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/storage/directio.py",
>> line 161, in write
>> 12:33:20     raise OSError(err, msg)
>> 12:33:20 OSError: [Errno 22] Invalid argument
>> 12:33:20 -------------------- >> begin captured logging << --------------------
>> 12:33:20 2016-09-28 05:32:03,254 DEBUG   [storage.PersistentDict]
>> (MainThread) Created a persistent dict with VGTagMetadataRW backend
>> 12:33:20 2016-09-28 05:32:03,255 DEBUG   [storage.PersistentDict]
>> (MainThread) read lines (VGTagMetadataRW)=[]
>> 12:33:20 2016-09-28 05:32:03,255 DEBUG   [storage.PersistentDict]
>> (MainThread) Empty metadata
>> 12:33:20 2016-09-28 05:32:03,255 DEBUG   [storage.PersistentDict]
>> (MainThread) Starting transaction
>> 12:33:20 2016-09-28 05:32:03,256 DEBUG   [storage.PersistentDict]
>> (MainThread) Flushing changes
>> 12:33:20 2016-09-28 05:32:03,256 DEBUG   [storage.PersistentDict]
>> (MainThread) about to write lines (VGTagMetadataRW)=['CLASS=Data',
>> 'POOL_UUID=52fe3782-ed7a-4d84-be3d-236faebdca2d',
>> 'SDUUID=c088020e-91b2-45e6-85fd-eea3fce58764', 'VERSION=3',
>> '_SHA_CKSUM=3f26f105271d3f7af6ea7458fb179418e7f9c139']
>> 12:33:20 2016-09-28 05:32:03,257 DEBUG
>> [storage.Metadata.VGTagMetadataRW] (MainThread) Updating metadata
>> adding=MDT_POOL_UUID=52fe3782-ed7a-4d84-be3d-236faebdca2d,
>> MDT__SHA_CKSUM=3f26f105271d3f7af6ea7458fb179418e7f9c139,
>> MDT_SDUUID=c088020e-91b2-45e6-85fd-eea3fce58764, MDT_CLASS=Data,
>> MDT_VERSION=3 removing=
>> 12:33:20 2016-09-28 05:32:03,257 DEBUG   [storage.PersistentDict]
>> (MainThread) Finished transaction
>> 12:33:20 2016-09-28 05:32:03,261 INFO
>> [storage.BlockVolumeArtifacts] (MainThread) Create placeholder
>> /var/tmp/tmp3UKYqC/52fe3782-ed7a-4d84-be3d-236faebdca2d/c088020e-91b2-45e6-85fd-eea3fce58764/images/a1e3aa74-2304-4e0c-b8e8-f5b86f8d3ac7
>> for image's volumes
>> 12:33:20 2016-09-28 05:32:03,264 WARNING
>> [storage.StorageDomainManifest] (MainThread) Could not find mapping
>> for lv c088020e-91b2-45e6-85fd-eea3fce58764/3267d54d-c0fa-4fe1-b82b-b88dd5f90de3
>> 12:33:20 2016-09-28 05:32:03,264 DEBUG
>> [storage.StorageDomainManifest] (MainThread) Found freeSlot 4 in VG
>> c088020e-91b2-45e6-85fd-eea3fce58764
>> 12:33:20 --------------------- >> end captured logging << ---------------------
>> 1
>> _______________________________________________
>> Infra mailing list
>> Infra at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>
>
>
> --
> Barak Korren
> bkorren at redhat.com
> RHEV-CI Team