[ovirt-devel] [VDSM] All tests using directio fail on CI

Wed Sep 28 20:39:27 UTC 2016

On Sep 28, 2016 11:37 PM, "Nir Soffer" <nsoffer at redhat.com> wrote:
>
> On Wed, Sep 28, 2016 at 11:20 PM, Nir Soffer <nsoffer at redhat.com> wrote:
> > On Wed, Sep 28, 2016 at 10:31 PM, Barak Korren <bkorren at redhat.com>
wrote:
> >> The CI setup did not change recently.
> >
> > Great
> >
> >> All standard-CI jobs run inside mock (chroot) which is stored on top
> >> of a regular FS, so they should not be affected by the slave OS at all
> >> as far as FS settings go.
> >>
> >> But perhaps some slave-OS/mock-OS combination is acting strangely, so
> >> could you be more specific and point to particular job runs that fail?
> >
> > This jobs failed, but it was deleted (I get 404 now):
> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/2530/
>
> Oops, wrong build.
>
> This is the failing build:
>
>
http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/1054/console
>
> And this is probably the reason - using a ram disk:
>
> 12:24:53 Building remotely on ovirt-srv08.phx.ovirt.org (phx physical
> integ-tests ram_disk fc23) in workspace
> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64
>
> We cannot run the storage tests using a ramdisk. We are creating
> (tiny) volumes and storage domains and doing copies, this code cannot
> work with ramdisk.

Will it work on zram?
What if we configure ram based iSCSI targets?
Y.

>
> Attaching full console log before the job is deleted.
>
> >
> > Then I triggered the tests and they passed:
> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/2548/
> >
> > Thanks
> > Nir
> >
> >>
> >> On 28 September 2016 at 22:00, Nir Soffer <nsoffer at redhat.com> wrote:
> >>> Hi all,
> >>>
> >>> It seems that the CI setup has changed, and /var/tmp is using now
tempfs.
> >>>
> >>> This is not compatible with vdsm tests, assuming that /var/tmp is a
real file
> >>> system. This is the reason we do not use /tmp.
> >>>
> >>> We have lot of storage tests using directio, and directio cannot work
on
> >>> tempfs.
> >>>
> >>> Please check the slaves and make sure /var/tmp is using file system
supporting
> >>> directio.
> >>>
> >>> See example failure bellow.
> >>>
> >>> Nir
> >>>
> >>> ----
> >>>
> >>> 12:33:20
======================================================================
> >>> 12:33:20 ERROR: test_create_fail_creating_lease
> >>> (storage_volume_artifacts_test.BlockVolumeArtifactsTests)
> >>> 12:33:20
----------------------------------------------------------------------
> >>> 12:33:20 Traceback (most recent call last):
> >>> 12:33:20   File
> >>>
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/storage_volume_artifacts_test.py",
> >>> line 485, in test_create_fail_creating_lease
> >>> 12:33:20     *BASE_PARAMS[sc.RAW_FORMAT])
> >>> 12:33:20   File "/usr/lib64/python2.7/unittest/case.py", line 513, in
> >>> assertRaises
> >>> 12:33:20     callableObj(*args, **kwargs)
> >>> 12:33:20   File
> >>>
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/sdm/volume_artifacts.py",
> >>> line 391, in create
> >>> 12:33:20     desc, parent)
> >>> 12:33:20   File
> >>>
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/sdm/volume_artifacts.py",
> >>> line 482, in _create_metadata
> >>> 12:33:20     sc.LEGAL_VOL)
> >>> 12:33:20   File
> >>>
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/volume.py",
> >>> line 427, in newMetadata
> >>> 12:33:20     cls.createMetadata(metaId, meta.legacy_info())
> >>> 12:33:20   File
> >>>
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/volume.py",
> >>> line 420, in createMetadata
> >>> 12:33:20     cls._putMetadata(metaId, meta)
> >>> 12:33:20   File
> >>>
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm/storage/blockVolume.py",
> >>> line 242, in _putMetadata
> >>> 12:33:20     f.write(data)
> >>> 12:33:20   File
> >>>
"/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/vdsm/storage/directio.py",
> >>> line 161, in write
> >>> 12:33:20     raise OSError(err, msg)
> >>> 12:33:20 OSError: [Errno 22] Invalid argument
> >>> 12:33:20 -------------------- >> begin captured logging <<
--------------------
> >>> 12:33:20 2016-09-28 05:32:03,254 DEBUG   [storage.PersistentDict]
> >>> (MainThread) Created a persistent dict with VGTagMetadataRW backend
> >>> 12:33:20 2016-09-28 05:32:03,255 DEBUG   [storage.PersistentDict]
> >>> (MainThread) read lines (VGTagMetadataRW)=[]
> >>> 12:33:20 2016-09-28 05:32:03,255 DEBUG   [storage.PersistentDict]
> >>> (MainThread) Empty metadata
> >>> 12:33:20 2016-09-28 05:32:03,255 DEBUG   [storage.PersistentDict]
> >>> (MainThread) Starting transaction
> >>> 12:33:20 2016-09-28 05:32:03,256 DEBUG   [storage.PersistentDict]
> >>> (MainThread) Flushing changes
> >>> 12:33:20 2016-09-28 05:32:03,256 DEBUG   [storage.PersistentDict]
> >>> (MainThread) about to write lines (VGTagMetadataRW)=['CLASS=Data',
> >>> 'POOL_UUID=52fe3782-ed7a-4d84-be3d-236faebdca2d',
> >>> 'SDUUID=c088020e-91b2-45e6-85fd-eea3fce58764', 'VERSION=3',
> >>> '_SHA_CKSUM=3f26f105271d3f7af6ea7458fb179418e7f9c139']
> >>> 12:33:20 2016-09-28 05:32:03,257 DEBUG
> >>> [storage.Metadata.VGTagMetadataRW] (MainThread) Updating metadata
> >>> adding=MDT_POOL_UUID=52fe3782-ed7a-4d84-be3d-236faebdca2d,
> >>> MDT__SHA_CKSUM=3f26f105271d3f7af6ea7458fb179418e7f9c139,
> >>> MDT_SDUUID=c088020e-91b2-45e6-85fd-eea3fce58764, MDT_CLASS=Data,
> >>> MDT_VERSION=3 removing=
> >>> 12:33:20 2016-09-28 05:32:03,257 DEBUG   [storage.PersistentDict]
> >>> (MainThread) Finished transaction
> >>> 12:33:20 2016-09-28 05:32:03,261 INFO
> >>> [storage.BlockVolumeArtifacts] (MainThread) Create placeholder
> >>>
/var/tmp/tmp3UKYqC/52fe3782-ed7a-4d84-be3d-236faebdca2d/c088020e-91b2-45e6-85fd-eea3fce58764/images/a1e3aa74-2304-4e0c-b8e8-f5b86f8d3ac7
> >>> for image's volumes
> >>> 12:33:20 2016-09-28 05:32:03,264 WARNING
> >>> [storage.StorageDomainManifest] (MainThread) Could not find mapping
> >>> for lv
c088020e-91b2-45e6-85fd-eea3fce58764/3267d54d-c0fa-4fe1-b82b-b88dd5f90de3
> >>> 12:33:20 2016-09-28 05:32:03,264 DEBUG
> >>> [storage.StorageDomainManifest] (MainThread) Found freeSlot 4 in VG
> >>> c088020e-91b2-45e6-85fd-eea3fce58764
> >>> 12:33:20 --------------------- >> end captured logging <<
---------------------
> >>> 1
> >>> _______________________________________________
> >>> Infra mailing list
> >>> Infra at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/infra
> >>
> >>
> >>
> >> --
> >> Barak Korren
> >> bkorren at redhat.com
> >> RHEV-CI Team
>
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20160928/210cba36/attachment.html>