Hey, I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code. That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level). Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features. [0] https://gerrit.ovirt.org/#/c/89830/ mpolednik
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' ) On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com> wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
[0] https://gerrit.ovirt.org/#/c/89830/
mpolednik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch. Such changes require testing with all storage types we support. Nir On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
[0] https://gerrit.ovirt.org/#/c/89830/
mpolednik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Well, we already have HE suite that runs on ISCSI, so at least we have NFS+ISCSI on nested, for real storage testing, you'll have to do it manually
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
[0] https://gerrit.ovirt.org/#/c/89830/
mpolednik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Wed, Apr 11, 2018 at 12:38 PM Eyal Edri <eedri@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Well, we already have HE suite that runs on ISCSI, so at least we have NFS+ISCSI on nested, for real storage testing, you'll have to do it manually
We need glusterfs (both native and fuse based), and cinder/ceph storage. But we cannot practically test all flows with all types of storage for every patch. Nir
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
[0] https://gerrit.ovirt.org/#/c/89830/
mpolednik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ)
On 11/04/18 12:27 +0000, Nir Soffer wrote:
On Wed, Apr 11, 2018 at 12:38 PM Eyal Edri <eedri@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Well, we already have HE suite that runs on ISCSI, so at least we have NFS+ISCSI on nested, for real storage testing, you'll have to do it manually
We need glusterfs (both native and fuse based), and cinder/ceph storage.
But we cannot practically test all flows with all types of storage for every patch.
That leads to a question - how do I go around verifying such patch without sufficient environment? Is there someone from storage QA that could assist with this?
Nir
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
[0] https://gerrit.ovirt.org/#/c/89830/
mpolednik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Wed, Apr 11, 2018 at 3:30 PM Martin Polednik <mpolednik@redhat.com> wrote:
On 11/04/18 12:27 +0000, Nir Soffer wrote:
On Wed, Apr 11, 2018 at 12:38 PM Eyal Edri <eedri@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Well, we already have HE suite that runs on ISCSI, so at least we have NFS+ISCSI on nested, for real storage testing, you'll have to do it manually
We need glusterfs (both native and fuse based), and cinder/ceph storage.
But we cannot practically test all flows with all types of storage for every patch.
That leads to a question - how do I go around verifying such patch without sufficient environment? Is there someone from storage QA that could assist with this?
Good question! I hope Denis can help with verifying the glusterfs changes. With cinder/ceph, maybe Elad can provide a setup for testing, or run some automation tests on the patch? Elad also have other automated tests for NFS/iSCSI that are worth running before we merge such changes. Nir
Nir
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
[0] https://gerrit.ovirt.org/#/c/89830/
mpolednik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. < https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Wed, Apr 11, 2018 at 3:27 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:38 PM Eyal Edri <eedri@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Well, we already have HE suite that runs on ISCSI, so at least we have NFS+ISCSI on nested, for real storage testing, you'll have to do it manually
We need glusterfs (both native and fuse based), and cinder/ceph storage.
We have Gluster in o-s-t as well, as part of the HC suite. It doesn't use Fuse though.
But we cannot practically test all flows with all types of storage for every patch.
Indeed. But we could add easily do some, and we should at least execute the minimal set that we are able to easily via o-s-t. Y.
Nir
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
[0] https://gerrit.ovirt.org/#/c/89830/
mpolednik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
In particular: dynamic_ownership was set to 0 prehistorically (as part of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because libvirt, running as root, was not able to play properly with root-squash nfs mounts.
Have you attempted this use case? I join to Nir's request to run this with storage QE.
+Elad On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
In particular: dynamic_ownership was set to 0 prehistorically (as part of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because libvirt, running as root, was not able to play properly with root-squash nfs mounts.
Have you attempted this use case?
I join to Nir's request to run this with storage QE.
-- Raz Tamir Manager, RHV QE
We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder, will have to check, since usually, we don't execute our automation on them. On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> wrote:
+Elad
On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
In particular: dynamic_ownership was set to 0 prehistorically (as part of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because libvirt, running as root, was not able to play properly with root-squash nfs mounts.
Have you attempted this use case?
I join to Nir's request to run this with storage QE.
--
Raz Tamir Manager, RHV QE
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder, will have to check, since usually, we don't execute our automation on them.
Any update on this? I believe the gluster tests were successful, OST passes fine and unit tests pass fine, that makes the storage backends test the last required piece.
On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> wrote:
+Elad
On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
In particular: dynamic_ownership was set to 0 prehistorically (as part of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because libvirt, running as root, was not able to play properly with root-squash nfs mounts.
Have you attempted this use case?
I join to Nir's request to run this with storage QE.
--
Raz Tamir Manager, RHV QE
Hi, sorry if I misunderstood, I waited for more input regarding what areas have to be tested here. On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <mpolednik@redhat.com> wrote:
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder, will have to check, since usually, we don't execute our automation on them.
Any update on this? I believe the gluster tests were successful, OST passes fine and unit tests pass fine, that makes the storage backends test the last required piece.
On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> wrote:
+Elad
On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible
before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com
wrote:
Hey, > > I've created a patch[0] that is finally able to activate libvirt's > dynamic_ownership for VDSM while not negatively affecting > functionality of our storage code. > > That of course comes with quite a bit of code removal, mostly in the > area of host devices, hwrng and anything that touches devices; bunch > of test changes and one XML generation caveat (storage is handled by > VDSM, therefore disk relabelling needs to be disabled on the VDSM > level). > > Because of the scope of the patch, I welcome storage/virt/network > people to review the code and consider the implication this change > has > on current/future features. > > [0] https://gerrit.ovirt.org/#/c/89830/ > > In particular: dynamic_ownership was set to 0 prehistorically (as
part of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because libvirt, running as root, was not able to play properly with root-squash nfs mounts.
Have you attempted this use case?
I join to Nir's request to run this with storage QE.
--
Raz Tamir Manager, RHV QE
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what areas have to be tested here.
I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity. mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <mpolednik@redhat.com> wrote:
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder, will have to check, since usually, we don't execute our automation on them.
Any update on this? I believe the gluster tests were successful, OST passes fine and unit tests pass fine, that makes the storage backends test the last required piece.
On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> wrote:
+Elad
On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible > before merging ( using 'ci please build' ) > > But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com >
> wrote: > > Hey, >> >> I've created a patch[0] that is finally able to activate libvirt's >> dynamic_ownership for VDSM while not negatively affecting >> functionality of our storage code. >> >> That of course comes with quite a bit of code removal, mostly in the >> area of host devices, hwrng and anything that touches devices; bunch >> of test changes and one XML generation caveat (storage is handled by >> VDSM, therefore disk relabelling needs to be disabled on the VDSM >> level). >> >> Because of the scope of the patch, I welcome storage/virt/network >> people to review the code and consider the implication this change >> has >> on current/future features. >> >> [0] https://gerrit.ovirt.org/#/c/89830/ >> >> > In particular: dynamic_ownership was set to 0 prehistorically (as
part of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because libvirt, running as root, was not able to play properly with root-squash nfs mounts.
Have you attempted this use case?
I join to Nir's request to run this with storage QE.
--
Raz Tamir Manager, RHV QE
Hi Martin, I see [1] requires a rebase, can you please take care? At the moment, our automation is stable only on iSCSI, NFS, Gluster and FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment. [1] https://gerrit.ovirt.org/#/c/89830/ Thanks On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what areas have to be tested here.
I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity.
mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <mpolednik@redhat.com>
wrote:
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder,
will have to check, since usually, we don't execute our automation on them.
Any update on this? I believe the gluster tests were successful, OST passes fine and unit tests pass fine, that makes the storage backends test the last required piece.
On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> wrote:
+Elad
On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
> > Please make sure to run as much OST suites on this patch as possible > >> before merging ( using 'ci please build' ) >> >> >> But note that OST is not a way to verify the patch. > > Such changes require testing with all storage types we support. > > Nir > > On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < > mpolednik@redhat.com > > > > wrote: >> >> Hey, >> >>> >>> I've created a patch[0] that is finally able to activate libvirt's >>> dynamic_ownership for VDSM while not negatively affecting >>> functionality of our storage code. >>> >>> That of course comes with quite a bit of code removal, mostly in >>> the >>> area of host devices, hwrng and anything that touches devices; >>> bunch >>> of test changes and one XML generation caveat (storage is handled >>> by >>> VDSM, therefore disk relabelling needs to be disabled on the VDSM >>> level). >>> >>> Because of the scope of the patch, I welcome storage/virt/network >>> people to review the code and consider the implication this change >>> has >>> on current/future features. >>> >>> [0] https://gerrit.ovirt.org/#/c/89830/ >>> >>> >>> In particular: dynamic_ownership was set to 0 prehistorically (as >> > part of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because libvirt, running as root, was not able to play properly with root-squash nfs mounts.
Have you attempted this use case?
I join to Nir's request to run this with storage QE.
--
Raz Tamir Manager, RHV QE
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what areas have to be tested here.
I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity.
mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <mpolednik@redhat.com>
wrote:
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder,
will have to check, since usually, we don't execute our automation on them.
Any update on this? I believe the gluster tests were successful, OST passes fine and unit tests pass fine, that makes the storage backends test the last required piece.
On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> wrote:
+Elad
On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
> > On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote: > >> >> Please make sure to run as much OST suites on this patch as possible >> >>> before merging ( using 'ci please build' ) >>> >>> >>> But note that OST is not a way to verify the patch. >> >> Such changes require testing with all storage types we support. >> >> Nir >> >> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >> mpolednik@redhat.com >> > >> >> wrote: >>> >>> Hey, >>> >>>> >>>> I've created a patch[0] that is finally able to activate libvirt's >>>> dynamic_ownership for VDSM while not negatively affecting >>>> functionality of our storage code. >>>> >>>> That of course comes with quite a bit of code removal, mostly in >>>> the >>>> area of host devices, hwrng and anything that touches devices; >>>> bunch >>>> of test changes and one XML generation caveat (storage is handled >>>> by >>>> VDSM, therefore disk relabelling needs to be disabled on the VDSM >>>> level). >>>> >>>> Because of the scope of the patch, I welcome storage/virt/network >>>> people to review the code and consider the implication this change >>>> has >>>> on current/future features. >>>> >>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>> >>>> >>>> In particular: dynamic_ownership was set to 0 prehistorically (as >>> >> part > of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because > libvirt, > running as root, was not able to play properly with root-squash nfs > mounts. > > Have you attempted this use case? > > I join to Nir's request to run this with storage QE. > > >
--
Raz Tamir Manager, RHV QE
Triggered a sanity tier1 execution [1] using [2], which covers all the requested areas, on iSCSI, NFS and Gluster. I'll update with the results. [1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4. 2_dev/job/rhv-4.2-ge-flow-storage/1161/ [2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64 On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and FC.
Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what
areas have to be tested here.
I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity.
mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <mpolednik@redhat.com>
wrote:
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder,
will have to check, since usually, we don't execute our automation on them.
Any update on this? I believe the gluster tests were successful, OST passes fine and unit tests pass fine, that makes the storage backends test the last required piece.
On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> wrote:
+Elad
> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> > wrote: > > On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> > wrote: > > >> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> >> wrote: >> >> >>> Please make sure to run as much OST suites on this patch as >>> possible >>> >>> before merging ( using 'ci please build' ) >>>> >>>> >>>> But note that OST is not a way to verify the patch. >>>> >>> >>> Such changes require testing with all storage types we support. >>> >>> Nir >>> >>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>> mpolednik@redhat.com >>> > >>> >>> wrote: >>> >>>> >>>> Hey, >>>> >>>> >>>>> I've created a patch[0] that is finally able to activate >>>>> libvirt's >>>>> dynamic_ownership for VDSM while not negatively affecting >>>>> functionality of our storage code. >>>>> >>>>> That of course comes with quite a bit of code removal, mostly in >>>>> the >>>>> area of host devices, hwrng and anything that touches devices; >>>>> bunch >>>>> of test changes and one XML generation caveat (storage is handled >>>>> by >>>>> VDSM, therefore disk relabelling needs to be disabled on the VDSM >>>>> level). >>>>> >>>>> Because of the scope of the patch, I welcome storage/virt/network >>>>> people to review the code and consider the implication this >>>>> change >>>>> has >>>>> on current/future features. >>>>> >>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>> >>>>> >>>>> In particular: dynamic_ownership was set to 0 prehistorically >>>>> (as >>>>> >>>> >>>> part >>> >> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because >> libvirt, >> running as root, was not able to play properly with root-squash nfs >> mounts. >> >> Have you attempted this use case? >> >> I join to Nir's request to run this with storage QE. >> >> >> >> > -- > > > Raz Tamir > Manager, RHV QE > > > >
Hi, I've triggered another execution [1] due to some issues I saw in the first which are not related to the patch. The success rate is 78% which is low comparing to tier1 executions with code from downstream builds (95-100% success rates) [2].
From what I could see so far, there is an issue with move and copy operations to and from Gluster domains. For example [3].
The logs are attached. [1] *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-ti... <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-tier1-after-upgrade/7/testReport/>* [2] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/ rhv-4.2-ge-runner-tier1-after-upgrade/7/ [3] 2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' from=: :ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935, task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51) 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in deleteImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503, in deleteImage raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) ImageDoesNotExistInSD: Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' 2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted: "Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835- 5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268 (task:1181) 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09 -ca40ff054d d4' (dispatcher:82) On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Triggered a sanity tier1 execution [1] using [2], which covers all the requested areas, on iSCSI, NFS and Gluster. I'll update with the results.
[1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2 _dev/job/rhv-4.2-ge-flow-storage/1161/
[2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64
On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and
FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what
areas have to be tested here.
I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity.
mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <mpolednik@redhat.com
wrote:
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder,
> will > have to check, since usually, we don't execute our automation on > them. > > > Any update on this? I believe the gluster tests were successful, OST passes fine and unit tests pass fine, that makes the storage backends test the last required piece.
On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> wrote:
> +Elad > > >> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> >> wrote: >> >> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> >> wrote: >> >> >>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> >>> wrote: >>> >>> >>>> Please make sure to run as much OST suites on this patch as >>>> possible >>>> >>>> before merging ( using 'ci please build' ) >>>>> >>>>> >>>>> But note that OST is not a way to verify the patch. >>>>> >>>> >>>> Such changes require testing with all storage types we support. >>>> >>>> Nir >>>> >>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>>> mpolednik@redhat.com >>>> > >>>> >>>> wrote: >>>> >>>>> >>>>> Hey, >>>>> >>>>> >>>>>> I've created a patch[0] that is finally able to activate >>>>>> libvirt's >>>>>> dynamic_ownership for VDSM while not negatively affecting >>>>>> functionality of our storage code. >>>>>> >>>>>> That of course comes with quite a bit of code removal, mostly in >>>>>> the >>>>>> area of host devices, hwrng and anything that touches devices; >>>>>> bunch >>>>>> of test changes and one XML generation caveat (storage is >>>>>> handled >>>>>> by >>>>>> VDSM, therefore disk relabelling needs to be disabled on the >>>>>> VDSM >>>>>> level). >>>>>> >>>>>> Because of the scope of the patch, I welcome >>>>>> storage/virt/network >>>>>> people to review the code and consider the implication this >>>>>> change >>>>>> has >>>>>> on current/future features. >>>>>> >>>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>>> >>>>>> >>>>>> In particular: dynamic_ownership was set to 0 prehistorically >>>>>> (as >>>>>> >>>>> >>>>> part >>>> >>> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because >>> libvirt, >>> running as root, was not able to play properly with root-squash nfs >>> mounts. >>> >>> Have you attempted this use case? >>> >>> I join to Nir's request to run this with storage QE. >>> >>> >>> >>> >> -- >> >> >> Raz Tamir >> Manager, RHV QE >> >> >> >>
Sorry, this is the new execution link: https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-st... On Mon, Apr 23, 2018 at 1:23 AM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Hi, I've triggered another execution [1] due to some issues I saw in the first which are not related to the patch.
The success rate is 78% which is low comparing to tier1 executions with code from downstream builds (95-100% success rates) [2].
From what I could see so far, there is an issue with move and copy operations to and from Gluster domains. For example [3].
The logs are attached.
[1] *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-ti... <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-tier1-after-upgrade/7/testReport/>*
[2] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv- 4.2-ge-runner-tier1-after-upgrade/7/
[3] 2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' from=: :ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935, task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51) 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in deleteImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503, in deleteImage raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) ImageDoesNotExistInSD: Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' 2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted: "Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835- 5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268 (task:1181) 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054d d4' (dispatcher:82)
On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Triggered a sanity tier1 execution [1] using [2], which covers all the requested areas, on iSCSI, NFS and Gluster. I'll update with the results.
[1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2 _dev/job/rhv-4.2-ge-flow-storage/1161/
[2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64
On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and
FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what
areas have to be tested here.
I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity.
mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik < mpolednik@redhat.com>
wrote:
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
> > We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder, > >> will >> have to check, since usually, we don't execute our automation on >> them. >> >> >> Any update on this? I believe the gluster tests were successful, OST > passes fine and unit tests pass fine, that makes the storage backends > test the last required piece. > > > On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> > wrote: > > >> +Elad >> >> >>> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com >>> > >>> wrote: >>> >>> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> >>> wrote: >>> >>> >>>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> >>>> wrote: >>>> >>>> >>>>> Please make sure to run as much OST suites on this patch as >>>>> possible >>>>> >>>>> before merging ( using 'ci please build' ) >>>>>> >>>>>> >>>>>> But note that OST is not a way to verify the patch. >>>>>> >>>>> >>>>> Such changes require testing with all storage types we support. >>>>> >>>>> Nir >>>>> >>>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>>>> mpolednik@redhat.com >>>>> > >>>>> >>>>> wrote: >>>>> >>>>>> >>>>>> Hey, >>>>>> >>>>>> >>>>>>> I've created a patch[0] that is finally able to activate >>>>>>> libvirt's >>>>>>> dynamic_ownership for VDSM while not negatively affecting >>>>>>> functionality of our storage code. >>>>>>> >>>>>>> That of course comes with quite a bit of code removal, mostly >>>>>>> in >>>>>>> the >>>>>>> area of host devices, hwrng and anything that touches devices; >>>>>>> bunch >>>>>>> of test changes and one XML generation caveat (storage is >>>>>>> handled >>>>>>> by >>>>>>> VDSM, therefore disk relabelling needs to be disabled on the >>>>>>> VDSM >>>>>>> level). >>>>>>> >>>>>>> Because of the scope of the patch, I welcome >>>>>>> storage/virt/network >>>>>>> people to review the code and consider the implication this >>>>>>> change >>>>>>> has >>>>>>> on current/future features. >>>>>>> >>>>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>>>> >>>>>>> >>>>>>> In particular: dynamic_ownership was set to 0 prehistorically >>>>>>> (as >>>>>>> >>>>>> >>>>>> part >>>>> >>>> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because >>>> libvirt, >>>> running as root, was not able to play properly with root-squash >>>> nfs >>>> mounts. >>>> >>>> Have you attempted this use case? >>>> >>>> I join to Nir's request to run this with storage QE. >>>> >>>> >>>> >>>> >>> -- >>> >>> >>> Raz Tamir >>> Manager, RHV QE >>> >>> >>> >>>
Also, snapshot preview failed (2nd snapshot): 2018-04-22 18:01:06,253+0300 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Volume.create succeeded in 0.84 seconds (__init__:311) 2018-04-22 18:01:06,261+0300 INFO (tasks/6) [storage.ThreadPool.WorkerThread] START task 6823d724-cb1b-4706-a58a-83428363cce5 (cmd=<bound method Task.commit of <vdsm.storage.task.Task instance at 0x7f1aac54fc68>>, args=None) (threadPool :208) 2018-04-22 18:01:06,906+0300 WARN (check/loop) [storage.asyncutils] Call <bound method DirectioChecker._check of <DirectioChecker /rhev/data-center/mnt/yellow-vdsb.qa.lab.tlv.redhat.com:_Storage__NFS_storage__l ocal__ge2__nfs__0/46d2fd2b-bdd0-40f5-be4c-0aaf2a629f1b/dom_md/metadata running next_check=4920812.91 at 0x7f1aac3ed790>> delayed by 0.51 seconds (asyncutils:138) 2018-04-22 18:01:07,082+0300 WARN (tasks/6) [storage.ResourceManager] Resource factory failed to create resource '01_img_7df9d2b2-52b5-4ac2-a9f0-a1d1e93eb6d2.095ad9d6-3154-449c-868c-f975dcdcb729'. Canceling request. (resourceManager:543 ) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceManager.py", line 539, in registerResource obj = namespaceObj.factory.createResource(name, lockType) File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceFactories.py", line 193, in createResource lockType) File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceFactories.py", line 122, in __getResourceCandidatesList imgUUID=resourceName) File "/usr/lib/python2.7/site-packages/vdsm/storage/image.py", line 198, in getChain uuidlist = volclass.getImageVolumes(sdUUID, imgUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 1537, in getImageVolumes return cls.manifestClass.getImageVolumes(sdUUID, imgUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line 337, in getImageVolumes if (sd.produceVolume(imgUUID, volid).getImage() == imgUUID): File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 438, in produceVolume volUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line 69, in __init__ volUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 86, in __init__ self.validate() File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 112, in validate self.validateVolumePath() File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line 129, in validateVolumePath raise se.VolumeDoesNotExist(self.volUUID) VolumeDoesNotExist: Volume does not exist: (u'a404bfc9-57ef-4dcc-9f1b-458dfb08ad74',) 2018-04-22 18:01:07,083+0300 WARN (tasks/6) [storage.ResourceManager.Request] (ResName='01_img_7df9d2b2-52b5-4ac2-a9f0-a1d1e93eb6d2.095ad9d6-3154-449c-868c-f975dcdcb729', ReqID='79c96e70-7334-4402-a390-dc87f939b7d2') Tried to cancel a p rocessed request (resourceManager:187) 2018-04-22 18:01:07,084+0300 ERROR (tasks/6) [storage.TaskManager.Task] (Task='6823d724-cb1b-4706-a58a-83428363cce5') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 336, in run return self.cmd(*self.argslist, **self.argsdict) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper return method(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1939, in createVolume with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceManager.py", line 1025, in acquireResource return _manager.acquireResource(namespace, name, lockType, timeout=timeout) File "/usr/lib/python2.7/site-packages/vdsm/storage/resourceManager.py", line 475, in acquireResource raise se.ResourceAcqusitionFailed() ResourceAcqusitionFailed: Could not acquire resource. Probably resource factory threw an exception.: () 2018-04-22 18:01:07,735+0300 INFO (tasks/6) [storage.ThreadPool.WorkerThread] FINISH task 6823d724-cb1b-4706-a58a-83428363cce5 (threadPool:210) *Steps from [1]:* *17:54:41* 2018-04-22 17:54:41,574 INFO Test Setup 2: Creating VM vm_TestCase11660_2217544157*17:54:55* 2018-04-22 17:54:55,593 INFO 049: storage/rhevmtests.storage.storage_snapshots.test_live_snapshot.TestCase11660.test_live_snapshot[glusterfs]*17:54:55* 2018-04-22 17:54:55,593 INFO Create a snapshot while VM is running*17:54:55* 2018-04-22 17:54:55,593 INFO STORAGE: GLUSTERFS*17:58:04* 2018-04-22 17:58:04,761 INFO Test Step 3: Start writing continuously on VM vm_TestCase11660_2217544157 via dd*17:58:35* 2018-04-22 17:58:35,334 INFO Test Step 4: Creating live snapshot on a VM vm_TestCase11660_2217544157*17:58:35* 2018-04-22 17:58:35,334 INFO Test Step 5: Adding new snapshot to VM vm_TestCase11660_2217544157 with all disks*17:58:35* 2018-04-22 17:58:35,337 INFO Test Step 6: Add snapshot to VM vm_TestCase11660_2217544157 with {'description': 'snap_TestCase11660_2217545559', 'wait': True}*17:59:26* 2018-04-22 17:59:26,179 INFO Test Step 7: Writing files to VM's vm_TestCase11660_2217544157 disk*18:00:33* 2018-04-22 18:00:33,117 INFO Test Step 8: Shutdown vm vm_TestCase11660_2217544157 with {'async': 'false'}*18:01:04* 2018-04-22 18:01:04,038 INFO Test Step 9: Previewing snapshot snap_TestCase11660_2217545559 on VM vm_TestCase11660_2217544157 [1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-st... On Mon, Apr 23, 2018 at 1:29 AM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Sorry, this is the new execution link: https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/ rhv-4.2-ge-runner-storage/1048/testReport/
On Mon, Apr 23, 2018 at 1:23 AM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Hi, I've triggered another execution [1] due to some issues I saw in the first which are not related to the patch.
The success rate is 78% which is low comparing to tier1 executions with code from downstream builds (95-100% success rates) [2].
From what I could see so far, there is an issue with move and copy operations to and from Gluster domains. For example [3].
The logs are attached.
[1] *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-ti... <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-tier1-after-upgrade/7/testReport/>*
[2] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv- 4.2-ge-runner-tier1-after-upgrade/7/
[3] 2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' from=: :ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935, task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51) 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in deleteImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503, in deleteImage raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) ImageDoesNotExistInSD: Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' 2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted: "Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835- 5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268 (task:1181) 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054d d4' (dispatcher:82)
On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Triggered a sanity tier1 execution [1] using [2], which covers all the requested areas, on iSCSI, NFS and Gluster. I'll update with the results.
[1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2 _dev/job/rhv-4.2-ge-flow-storage/1161/
[2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64
On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and
FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com
wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what > areas > have to be tested here. > > I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity.
mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik < mpolednik@redhat.com>
> wrote: > > On 11/04/18 16:52 +0300, Elad Ben Aharon wrote: > >> >> We can test this on iSCSI, NFS and GlusterFS. As for ceph and >> cinder, >> >>> will >>> have to check, since usually, we don't execute our automation on >>> them. >>> >>> >>> Any update on this? I believe the gluster tests were successful, >> OST >> passes fine and unit tests pass fine, that makes the storage >> backends >> test the last required piece. >> >> >> On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> >> wrote: >> >> >>> +Elad >>> >>> >>>> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg < >>>> danken@redhat.com> >>>> wrote: >>>> >>>> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> >>>> wrote: >>>> >>>> >>>>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> >>>>> wrote: >>>>> >>>>> >>>>>> Please make sure to run as much OST suites on this patch as >>>>>> possible >>>>>> >>>>>> before merging ( using 'ci please build' ) >>>>>>> >>>>>>> >>>>>>> But note that OST is not a way to verify the patch. >>>>>>> >>>>>> >>>>>> Such changes require testing with all storage types we support. >>>>>> >>>>>> Nir >>>>>> >>>>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>>>>> mpolednik@redhat.com >>>>>> > >>>>>> >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> Hey, >>>>>>> >>>>>>> >>>>>>>> I've created a patch[0] that is finally able to activate >>>>>>>> libvirt's >>>>>>>> dynamic_ownership for VDSM while not negatively affecting >>>>>>>> functionality of our storage code. >>>>>>>> >>>>>>>> That of course comes with quite a bit of code removal, mostly >>>>>>>> in >>>>>>>> the >>>>>>>> area of host devices, hwrng and anything that touches devices; >>>>>>>> bunch >>>>>>>> of test changes and one XML generation caveat (storage is >>>>>>>> handled >>>>>>>> by >>>>>>>> VDSM, therefore disk relabelling needs to be disabled on the >>>>>>>> VDSM >>>>>>>> level). >>>>>>>> >>>>>>>> Because of the scope of the patch, I welcome >>>>>>>> storage/virt/network >>>>>>>> people to review the code and consider the implication this >>>>>>>> change >>>>>>>> has >>>>>>>> on current/future features. >>>>>>>> >>>>>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>>>>> >>>>>>>> >>>>>>>> In particular: dynamic_ownership was set to 0 >>>>>>>> prehistorically (as >>>>>>>> >>>>>>> >>>>>>> part >>>>>> >>>>> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because >>>>> libvirt, >>>>> running as root, was not able to play properly with root-squash >>>>> nfs >>>>> mounts. >>>>> >>>>> Have you attempted this use case? >>>>> >>>>> I join to Nir's request to run this with storage QE. >>>>> >>>>> >>>>> >>>>> >>>> -- >>>> >>>> >>>> Raz Tamir >>>> Manager, RHV QE >>>> >>>> >>>> >>>>
On 23/04/18 01:23 +0300, Elad Ben Aharon wrote:
Hi, I've triggered another execution [1] due to some issues I saw in the first which are not related to the patch.
The success rate is 78% which is low comparing to tier1 executions with code from downstream builds (95-100% success rates) [2].
Could you run the current master (without the dynamic_ownership patch) so that we have viable comparision?
From what I could see so far, there is an issue with move and copy operations to and from Gluster domains. For example [3].
The logs are attached.
[1] *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-ti... <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-tier1-after-upgrade/7/testReport/>*
[2] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/ rhv-4.2-ge-runner-tier1-after-upgrade/7/
[3] 2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' from=: :ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935, task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51) 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in deleteImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503, in deleteImage raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) ImageDoesNotExistInSD: Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'
2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted: "Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835- 5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268 (task:1181) 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09 -ca40ff054d d4' (dispatcher:82)
On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Triggered a sanity tier1 execution [1] using [2], which covers all the requested areas, on iSCSI, NFS and Gluster. I'll update with the results.
[1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2 _dev/job/rhv-4.2-ge-flow-storage/1161/
[2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64
On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and
FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what
areas have to be tested here.
I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity.
mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <mpolednik@redhat.com
wrote:
On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
> > We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder, > >> will >> have to check, since usually, we don't execute our automation on >> them. >> >> >> Any update on this? I believe the gluster tests were successful, OST > passes fine and unit tests pass fine, that makes the storage backends > test the last required piece. > > > On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> > wrote: > > >> +Elad >> >> >>> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com> >>> wrote: >>> >>> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> >>> wrote: >>> >>> >>>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> >>>> wrote: >>>> >>>> >>>>> Please make sure to run as much OST suites on this patch as >>>>> possible >>>>> >>>>> before merging ( using 'ci please build' ) >>>>>> >>>>>> >>>>>> But note that OST is not a way to verify the patch. >>>>>> >>>>> >>>>> Such changes require testing with all storage types we support. >>>>> >>>>> Nir >>>>> >>>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>>>> mpolednik@redhat.com >>>>> > >>>>> >>>>> wrote: >>>>> >>>>>> >>>>>> Hey, >>>>>> >>>>>> >>>>>>> I've created a patch[0] that is finally able to activate >>>>>>> libvirt's >>>>>>> dynamic_ownership for VDSM while not negatively affecting >>>>>>> functionality of our storage code. >>>>>>> >>>>>>> That of course comes with quite a bit of code removal, mostly in >>>>>>> the >>>>>>> area of host devices, hwrng and anything that touches devices; >>>>>>> bunch >>>>>>> of test changes and one XML generation caveat (storage is >>>>>>> handled >>>>>>> by >>>>>>> VDSM, therefore disk relabelling needs to be disabled on the >>>>>>> VDSM >>>>>>> level). >>>>>>> >>>>>>> Because of the scope of the patch, I welcome >>>>>>> storage/virt/network >>>>>>> people to review the code and consider the implication this >>>>>>> change >>>>>>> has >>>>>>> on current/future features. >>>>>>> >>>>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>>>> >>>>>>> >>>>>>> In particular: dynamic_ownership was set to 0 prehistorically >>>>>>> (as >>>>>>> >>>>>> >>>>>> part >>>>> >>>> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because >>>> libvirt, >>>> running as root, was not able to play properly with root-squash nfs >>>> mounts. >>>> >>>> Have you attempted this use case? >>>> >>>> I join to Nir's request to run this with storage QE. >>>> >>>> >>>> >>>> >>> -- >>> >>> >>> Raz Tamir >>> Manager, RHV QE >>> >>> >>> >>>
I will update with the results of the next tier1 execution on latest 4.2.3 On Mon, Apr 23, 2018 at 3:56 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 23/04/18 01:23 +0300, Elad Ben Aharon wrote:
Hi, I've triggered another execution [1] due to some issues I saw in the first which are not related to the patch.
The success rate is 78% which is low comparing to tier1 executions with code from downstream builds (95-100% success rates) [2].
Could you run the current master (without the dynamic_ownership patch) so that we have viable comparision?
From what I could see so far, there is an issue with move and copy
operations to and from Gluster domains. For example [3].
The logs are attached.
[1] *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv -4.2-ge-runner-tier1-after-upgrade/7/testReport/ <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv -4.2-ge-runner-tier1-after-upgrade/7/testReport/>*
[2] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/
rhv-4.2-ge-runner-tier1-after-upgrade/7/
[3] 2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' from=: :ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935, task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51) 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in deleteImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503, in deleteImage raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) ImageDoesNotExistInSD: Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'
2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted: "Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835- 5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268 (task:1181) 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09 -ca40ff054d d4' (dispatcher:82)
On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Triggered a sanity tier1 execution [1] using [2], which covers all the
requested areas, on iSCSI, NFS and Gluster. I'll update with the results.
[1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2 _dev/job/rhv-4.2-ge-flow-storage/1161/
[2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64
On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and
FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com
wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
Hi, sorry if I misunderstood, I waited for more input regarding what
> areas > have to be tested here. > > > I'd say that you have quite a bit of freedom in this regard. GlusterFS should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite that covers basic operations (start & stop VM, migrate it), snapshots and merging them, and whatever else would be important for storage sanity.
mpolednik
On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik < mpolednik@redhat.com >
wrote: > > On 11/04/18 16:52 +0300, Elad Ben Aharon wrote: > > >> We can test this on iSCSI, NFS and GlusterFS. As for ceph and >> cinder, >> >> will >>> have to check, since usually, we don't execute our automation on >>> them. >>> >>> >>> Any update on this? I believe the gluster tests were successful, >>> OST >>> >> passes fine and unit tests pass fine, that makes the storage >> backends >> test the last required piece. >> >> >> On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> >> wrote: >> >> >> +Elad >>> >>> >>> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com >>>> > >>>> wrote: >>>> >>>> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> >>>> wrote: >>>> >>>> >>>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> >>>>> wrote: >>>>> >>>>> >>>>> Please make sure to run as much OST suites on this patch as >>>>>> possible >>>>>> >>>>>> before merging ( using 'ci please build' ) >>>>>> >>>>>>> >>>>>>> >>>>>>> But note that OST is not a way to verify the patch. >>>>>>> >>>>>>> >>>>>> Such changes require testing with all storage types we support. >>>>>> >>>>>> Nir >>>>>> >>>>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>>>>> mpolednik@redhat.com >>>>>> > >>>>>> >>>>>> wrote: >>>>>> >>>>>> >>>>>>> Hey, >>>>>>> >>>>>>> >>>>>>> I've created a patch[0] that is finally able to activate >>>>>>>> libvirt's >>>>>>>> dynamic_ownership for VDSM while not negatively affecting >>>>>>>> functionality of our storage code. >>>>>>>> >>>>>>>> That of course comes with quite a bit of code removal, mostly >>>>>>>> in >>>>>>>> the >>>>>>>> area of host devices, hwrng and anything that touches devices; >>>>>>>> bunch >>>>>>>> of test changes and one XML generation caveat (storage is >>>>>>>> handled >>>>>>>> by >>>>>>>> VDSM, therefore disk relabelling needs to be disabled on the >>>>>>>> VDSM >>>>>>>> level). >>>>>>>> >>>>>>>> Because of the scope of the patch, I welcome >>>>>>>> storage/virt/network >>>>>>>> people to review the code and consider the implication this >>>>>>>> change >>>>>>>> has >>>>>>>> on current/future features. >>>>>>>> >>>>>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>>>>> >>>>>>>> >>>>>>>> In particular: dynamic_ownership was set to 0 prehistorically >>>>>>>> (as >>>>>>>> >>>>>>>> >>>>>>> part >>>>>>> >>>>>> >>>>>> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because >>>>> libvirt, >>>>> running as root, was not able to play properly with root-squash >>>>> nfs >>>>> mounts. >>>>> >>>>> Have you attempted this use case? >>>>> >>>>> I join to Nir's request to run this with storage QE. >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>> >>>> >>>> Raz Tamir >>>> Manager, RHV QE >>>> >>>> >>>> >>>> >>>>
Here it is - https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/ rhv-4.2-ge-runner-tier1/122/ This was executed over iscsi, nfs, gluster and fcp. It ended up with 98.43% success rate. 2 storage related test cases failed and their failures, from first glance, don't seem to be bugs. On Tue, Apr 24, 2018 at 12:37 AM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
I will update with the results of the next tier1 execution on latest 4.2.3
On Mon, Apr 23, 2018 at 3:56 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 23/04/18 01:23 +0300, Elad Ben Aharon wrote:
Hi, I've triggered another execution [1] due to some issues I saw in the first which are not related to the patch.
The success rate is 78% which is low comparing to tier1 executions with code from downstream builds (95-100% success rates) [2].
Could you run the current master (without the dynamic_ownership patch) so that we have viable comparision?
From what I could see so far, there is an issue with move and copy
operations to and from Gluster domains. For example [3].
The logs are attached.
[1] *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv -4.2-ge-runner-tier1-after-upgrade/7/testReport/ <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv -4.2-ge-runner-tier1-after-upgrade/7/testReport/>*
[2] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/
rhv-4.2-ge-runner-tier1-after-upgrade/7/
[3] 2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' from=: :ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935, task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51) 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in deleteImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503, in deleteImage raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) ImageDoesNotExistInSD: Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'
2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted: "Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835- 5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268 (task:1181) 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09 -ca40ff054d d4' (dispatcher:82)
On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Triggered a sanity tier1 execution [1] using [2], which covers all the
requested areas, on iSCSI, NFS and Gluster. I'll update with the results.
[1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2 _dev/job/rhv-4.2-ge-flow-storage/1161/
[2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64
On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and
FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik < mpolednik@redhat.com> wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
> > Hi, sorry if I misunderstood, I waited for more input regarding what > >> areas >> have to be tested here. >> >> >> I'd say that you have quite a bit of freedom in this regard. > GlusterFS > should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite > that covers basic operations (start & stop VM, migrate it), snapshots > and merging them, and whatever else would be important for storage > sanity. > > mpolednik > > > On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik < > mpolednik@redhat.com > > > > wrote: >> >> On 11/04/18 16:52 +0300, Elad Ben Aharon wrote: >> >> >>> We can test this on iSCSI, NFS and GlusterFS. As for ceph and >>> cinder, >>> >>> will >>>> have to check, since usually, we don't execute our automation on >>>> them. >>>> >>>> >>>> Any update on this? I believe the gluster tests were successful, >>>> OST >>>> >>> passes fine and unit tests pass fine, that makes the storage >>> backends >>> test the last required piece. >>> >>> >>> On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> >>> wrote: >>> >>> >>> +Elad >>>> >>>> >>>> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg < >>>>> danken@redhat.com> >>>>> wrote: >>>>> >>>>> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com >>>>> > >>>>> wrote: >>>>> >>>>> >>>>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> >>>>>> wrote: >>>>>> >>>>>> >>>>>> Please make sure to run as much OST suites on this patch as >>>>>>> possible >>>>>>> >>>>>>> before merging ( using 'ci please build' ) >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> But note that OST is not a way to verify the patch. >>>>>>>> >>>>>>>> >>>>>>> Such changes require testing with all storage types we support. >>>>>>> >>>>>>> Nir >>>>>>> >>>>>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>>>>>> mpolednik@redhat.com >>>>>>> > >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>>> Hey, >>>>>>>> >>>>>>>> >>>>>>>> I've created a patch[0] that is finally able to activate >>>>>>>>> libvirt's >>>>>>>>> dynamic_ownership for VDSM while not negatively affecting >>>>>>>>> functionality of our storage code. >>>>>>>>> >>>>>>>>> That of course comes with quite a bit of code removal, >>>>>>>>> mostly in >>>>>>>>> the >>>>>>>>> area of host devices, hwrng and anything that touches >>>>>>>>> devices; >>>>>>>>> bunch >>>>>>>>> of test changes and one XML generation caveat (storage is >>>>>>>>> handled >>>>>>>>> by >>>>>>>>> VDSM, therefore disk relabelling needs to be disabled on the >>>>>>>>> VDSM >>>>>>>>> level). >>>>>>>>> >>>>>>>>> Because of the scope of the patch, I welcome >>>>>>>>> storage/virt/network >>>>>>>>> people to review the code and consider the implication this >>>>>>>>> change >>>>>>>>> has >>>>>>>>> on current/future features. >>>>>>>>> >>>>>>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>>>>>> >>>>>>>>> >>>>>>>>> In particular: dynamic_ownership was set to 0 >>>>>>>>> prehistorically >>>>>>>>> (as >>>>>>>>> >>>>>>>>> >>>>>>>> part >>>>>>>> >>>>>>> >>>>>>> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) >>>>>> because >>>>>> libvirt, >>>>>> running as root, was not able to play properly with root-squash >>>>>> nfs >>>>>> mounts. >>>>>> >>>>>> Have you attempted this use case? >>>>>> >>>>>> I join to Nir's request to run this with storage QE. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>> >>>>> >>>>> Raz Tamir >>>>> Manager, RHV QE >>>>> >>>>> >>>>> >>>>> >>>>>
On 24/04/18 00:37 +0300, Elad Ben Aharon wrote:
I will update with the results of the next tier1 execution on latest 4.2.3
That isn't master but old branch though. Could you run it against *current* VDSM master?
On Mon, Apr 23, 2018 at 3:56 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 23/04/18 01:23 +0300, Elad Ben Aharon wrote:
Hi, I've triggered another execution [1] due to some issues I saw in the first which are not related to the patch.
The success rate is 78% which is low comparing to tier1 executions with code from downstream builds (95-100% success rates) [2].
Could you run the current master (without the dynamic_ownership patch) so that we have viable comparision?
From what I could see so far, there is an issue with move and copy
operations to and from Gluster domains. For example [3].
The logs are attached.
[1] *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv -4.2-ge-runner-tier1-after-upgrade/7/testReport/ <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv -4.2-ge-runner-tier1-after-upgrade/7/testReport/>*
[2] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/
rhv-4.2-ge-runner-tier1-after-upgrade/7/
[3] 2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' from=: :ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935, task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51) 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in deleteImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503, in deleteImage raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) ImageDoesNotExistInSD: Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'
2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted: "Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835- 5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268 (task:1181) 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH deleteImage error=Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-5f603e682f33, domain=e5fd29c8-52ba-467e-be09 -ca40ff054d d4' (dispatcher:82)
On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar@redhat.com> wrote:
Triggered a sanity tier1 execution [1] using [2], which covers all the
requested areas, on iSCSI, NFS and Gluster. I'll update with the results.
[1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2 _dev/job/rhv-4.2-ge-flow-storage/1161/
[2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64
On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
Hi Martin,
I see [1] requires a rebase, can you please take care?
Should be rebased.
At the moment, our automation is stable only on iSCSI, NFS, Gluster and
FC. Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's not stable enough at the moment.
That is still pretty good.
[1] https://gerrit.ovirt.org/#/c/89830/
Thanks
On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com > wrote:
On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
> > Hi, sorry if I misunderstood, I waited for more input regarding what > >> areas >> have to be tested here. >> >> >> I'd say that you have quite a bit of freedom in this regard. > GlusterFS > should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite > that covers basic operations (start & stop VM, migrate it), snapshots > and merging them, and whatever else would be important for storage > sanity. > > mpolednik > > > On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik < > mpolednik@redhat.com > > > > wrote: >> >> On 11/04/18 16:52 +0300, Elad Ben Aharon wrote: >> >> >>> We can test this on iSCSI, NFS and GlusterFS. As for ceph and >>> cinder, >>> >>> will >>>> have to check, since usually, we don't execute our automation on >>>> them. >>>> >>>> >>>> Any update on this? I believe the gluster tests were successful, >>>> OST >>>> >>> passes fine and unit tests pass fine, that makes the storage >>> backends >>> test the last required piece. >>> >>> >>> On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com> >>> wrote: >>> >>> >>> +Elad >>>> >>>> >>>> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com >>>>> > >>>>> wrote: >>>>> >>>>> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> >>>>> wrote: >>>>> >>>>> >>>>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> >>>>>> wrote: >>>>>> >>>>>> >>>>>> Please make sure to run as much OST suites on this patch as >>>>>>> possible >>>>>>> >>>>>>> before merging ( using 'ci please build' ) >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> But note that OST is not a way to verify the patch. >>>>>>>> >>>>>>>> >>>>>>> Such changes require testing with all storage types we support. >>>>>>> >>>>>>> Nir >>>>>>> >>>>>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>>>>>> mpolednik@redhat.com >>>>>>> > >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>>> Hey, >>>>>>>> >>>>>>>> >>>>>>>> I've created a patch[0] that is finally able to activate >>>>>>>>> libvirt's >>>>>>>>> dynamic_ownership for VDSM while not negatively affecting >>>>>>>>> functionality of our storage code. >>>>>>>>> >>>>>>>>> That of course comes with quite a bit of code removal, mostly >>>>>>>>> in >>>>>>>>> the >>>>>>>>> area of host devices, hwrng and anything that touches devices; >>>>>>>>> bunch >>>>>>>>> of test changes and one XML generation caveat (storage is >>>>>>>>> handled >>>>>>>>> by >>>>>>>>> VDSM, therefore disk relabelling needs to be disabled on the >>>>>>>>> VDSM >>>>>>>>> level). >>>>>>>>> >>>>>>>>> Because of the scope of the patch, I welcome >>>>>>>>> storage/virt/network >>>>>>>>> people to review the code and consider the implication this >>>>>>>>> change >>>>>>>>> has >>>>>>>>> on current/future features. >>>>>>>>> >>>>>>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>>>>>> >>>>>>>>> >>>>>>>>> In particular: dynamic_ownership was set to 0 prehistorically >>>>>>>>> (as >>>>>>>>> >>>>>>>>> >>>>>>>> part >>>>>>>> >>>>>>> >>>>>>> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because >>>>>> libvirt, >>>>>> running as root, was not able to play properly with root-squash >>>>>> nfs >>>>>> mounts. >>>>>> >>>>>> Have you attempted this use case? >>>>>> >>>>>> I join to Nir's request to run this with storage QE. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>> >>>>> >>>>> Raz Tamir >>>>> Manager, RHV QE >>>>> >>>>> >>>>> >>>>> >>>>>
On 27 Apr 2018, at 09:23, Martin Polednik <mpolednik@redhat.com> = wrote: =20 On 24/04/18 00:37 +0300, Elad Ben Aharon wrote:
I will update with the results of the next tier1 execution on latest = 4.2.3 =20 That isn't master but old branch though. Could you run it against *current* VDSM master? =20 On Mon, Apr 23, 2018 at 3:56 PM, Martin Polednik = <mpolednik@redhat.com> wrote: =20
On 23/04/18 01:23 +0300, Elad Ben Aharon wrote: =20
Hi, I've triggered another execution [1] due to some issues I saw = in the first which are not related to the patch. =20 The success rate is 78% which is low comparing to tier1 executions = with code from downstream builds (95-100% success rates) [2]. =20 =20 Could you run the current master (without the dynamic_ownership =
so that we have viable comparision? =20 =46rom what I could see so far, there is an issue with move and copy
operations to and from Gluster domains. For example [3]. =20 The logs are attached. =20 =20 [1] *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv -4.2-ge-runner-tier1-after-upgrade/7/testReport/ <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv -4.2-ge-runner-tier1-after-upgrade/7/testReport/>* =20 =20 =20 [2] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/ =20 rhv-4.2-ge-runner-tier1-after-upgrade/7/ =20 =20 =20 [3] 2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH deleteImage error=3DImage does not exist in domain: 'image=3Dcabb8846-7a4b-4244-9835-5f603e682f33, domain=3De5fd29c8-52ba-467e-be09-ca40ff054dd4' from=3D: :ffff:10.35.161.182,40936, = flow_id=3Ddisks_syncAction_ba6b2630-5976-4935, task_id=3D3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51) 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) = [storage.TaskManager.Task] (Task=3D'3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error = (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line = 882, in _run return fn(*args, **kargs) File "<string>", line 2, in deleteImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line = 49, in method ret =3D func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line = 1503, in deleteImage raise se.ImageDoesNotExistInSD(imgUUID, sdUUID) ImageDoesNotExistInSD: Image does not exist in domain: 'image=3Dcabb8846-7a4b-4244-9835-5f603e682f33, domain=3De5fd29c8-52ba-467e-be09-ca40ff054dd4' =20 2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) = [storage.TaskManager.Task] (Task=3D'3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is = aborted: "Image does not exist in domain: 'image=3Dcabb8846-7a4b-4244-9835- 5f603e682f33, domain=3De5fd29c8-52ba-467e-be09-ca40ff054dd4'" - = code 268 (task:1181) 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] = FINISH deleteImage error=3DImage does not exist in domain: 'image=3Dcabb8846-7a4b-4244-9835-5f603e682f33, domain=3De5fd29c8-52ba-467e-be09 -ca40ff054d d4' (dispatcher:82) =20 =20 =20 On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon = <ebenahar@redhat.com> wrote: =20 Triggered a sanity tier1 execution [1] using [2], which covers all =
requested areas, on iSCSI, NFS and Gluster. I'll update with the results. =20 [1] https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2 _dev/job/rhv-4.2-ge-flow-storage/1161/ =20 [2] https://gerrit.ovirt.org/#/c/89830/ vdsm-4.30.0-291.git77aef9a.el7.x86_64 =20 =20 =20 On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik = <mpolednik@redhat.com> wrote: =20 On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
=20 Hi Martin, >=20 > I see [1] requires a rebase, can you please take care? >=20 >=20 Should be rebased. =20 At the moment, our automation is stable only on iSCSI, NFS, = Gluster and =20 > FC. > Ceph is not supported and Cinder will be stabilized soon, AFAIR, = it's > not > stable enough at the moment. >=20 >=20 That is still pretty good. =20 =20 [1] https://gerrit.ovirt.org/#/c/89830/ =20 >=20 >=20 > Thanks >=20 > On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik = <mpolednik@redhat.com > > > wrote: >=20 > On 18/04/18 11:37 +0300, Elad Ben Aharon wrote: >=20 >>=20 >> Hi, sorry if I misunderstood, I waited for more input regarding = what >>=20 >>> areas >>> have to be tested here. >>>=20 >>>=20 >>> I'd say that you have quite a bit of freedom in this regard. >> GlusterFS >> should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some = suite >> that covers basic operations (start & stop VM, migrate it), = snapshots >> and merging them, and whatever else would be important for = storage >> sanity. >>=20 >> mpolednik >>=20 >>=20 >> On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik < >> mpolednik@redhat.com >> > >>=20 >> wrote: >>>=20 >>> On 11/04/18 16:52 +0300, Elad Ben Aharon wrote: >>>=20 >>>=20 >>>> We can test this on iSCSI, NFS and GlusterFS. As for ceph and >>>> cinder, >>>>=20 >>>> will >>>>> have to check, since usually, we don't execute our = automation on >>>>> them. >>>>>=20 >>>>>=20 >>>>> Any update on this? I believe the gluster tests were = successful, >>>>> OST >>>>>=20 >>>> passes fine and unit tests pass fine, that makes the storage >>>> backends >>>> test the last required piece. >>>>=20 >>>>=20 >>>> On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir = <ratamir@redhat.com> >>>> wrote: >>>>=20 >>>>=20 >>>> +Elad >>>>>=20 >>>>>=20 >>>>> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg = <danken@redhat.com >>>>>> > >>>>>> wrote: >>>>>>=20 >>>>>> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer = <nsoffer@redhat.com> >>>>>> wrote: >>>>>>=20 >>>>>>=20 >>>>>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri = <eedri@redhat.com> >>>>>>> wrote: >>>>>>>=20 >>>>>>>=20 >>>>>>> Please make sure to run as much OST suites on this patch = as >>>>>>>> possible >>>>>>>>=20 >>>>>>>> before merging ( using 'ci please build' ) >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> But note that OST is not a way to verify the patch. >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>> Such changes require testing with all storage types we = support. >>>>>>>>=20 >>>>>>>> Nir >>>>>>>>=20 >>>>>>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik < >>>>>>>> mpolednik@redhat.com >>>>>>>> > >>>>>>>>=20 >>>>>>>> wrote: >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>> Hey, >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> I've created a patch[0] that is finally able to activate >>>>>>>>>> libvirt's >>>>>>>>>> dynamic_ownership for VDSM while not negatively = affecting >>>>>>>>>> functionality of our storage code. >>>>>>>>>>=20 >>>>>>>>>> That of course comes with quite a bit of code removal, = mostly >>>>>>>>>> in >>>>>>>>>> the >>>>>>>>>> area of host devices, hwrng and anything that touches = devices; >>>>>>>>>> bunch >>>>>>>>>> of test changes and one XML generation caveat (storage = is >>>>>>>>>> handled >>>>>>>>>> by >>>>>>>>>> VDSM, therefore disk relabelling needs to be disabled = on the >>>>>>>>>> VDSM >>>>>>>>>> level). >>>>>>>>>>=20 >>>>>>>>>> Because of the scope of the patch, I welcome >>>>>>>>>> storage/virt/network >>>>>>>>>> people to review the code and consider the implication =
>>>>>>>>>> change >>>>>>>>>> has >>>>>>>>>> on current/future features. >>>>>>>>>>=20 >>>>>>>>>> [0] https://gerrit.ovirt.org/#/c/89830/ >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> In particular: dynamic_ownership was set to 0 =
--Apple-Mail=_807B5345-431A-48DA-A469-7819B1EAE0EF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Elad, to make it easier to compare, Martin backported the change to 4.2 so it = is actually comparable with a run without that patch. Would you please = try that out?=20 It would be best to have 4.2 upstream and this[1] run to really minimize = the noise. Thanks, michal [1] = http://jenkins.ovirt.org/job/vdsm_4.2_build-artifacts-on-demand-el7-x86_64= /28/ = <http://jenkins.ovirt.org/job/vdsm_4.2_build-artifacts-on-demand-el7-x86_6= 4/28/> patch) the this prehistorically
>>>>>>>>>> (as >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>> part >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>> of https://bugzilla.redhat.com/show_bug.cgi?id=3D554961 ) = because >>>>>>> libvirt, >>>>>>> running as root, was not able to play properly with = root-squash >>>>>>> nfs >>>>>>> mounts. >>>>>>>=20 >>>>>>> Have you attempted this use case? >>>>>>>=20 >>>>>>> I join to Nir's request to run this with storage QE. >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> -- >>>>>>=20 >>>>>>=20 >>>>>> Raz Tamir >>>>>> Manager, RHV QE >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 =20 =20 =20
Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel =20 =20
--Apple-Mail=_807B5345-431A-48DA-A469-7819B1EAE0EF Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D"">Hi = Elad,<div class=3D"">to make it easier to compare, Martin backported the = change to 4.2 so it is actually comparable with a run without that = patch. Would you please try that out? </div><div class=3D"">It = would be best to have 4.2 upstream and this[1] run to really minimize = the noise.</div><div class=3D""><br class=3D""></div><div = class=3D"">Thanks,</div><div class=3D"">michal</div><div class=3D""><br = class=3D""></div><div class=3D"">[1] <a = href=3D"http://jenkins.ovirt.org/job/vdsm_4.2_build-artifacts-on-demand-el= 7-x86_64/28/" = class=3D"">http://jenkins.ovirt.org/job/vdsm_4.2_build-artifacts-on-demand= -el7-x86_64/28/</a></div><div class=3D""><br class=3D""><div><blockquote = type=3D"cite" class=3D""><div class=3D"">On 27 Apr 2018, at 09:23, = Martin Polednik <<a href=3D"mailto:mpolednik@redhat.com" = class=3D"">mpolednik@redhat.com</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><div class=3D"">On = 24/04/18 00:37 +0300, Elad Ben Aharon wrote:<br class=3D""><blockquote = type=3D"cite" class=3D"">I will update with the results of the next = tier1 execution on latest 4.2.3<br class=3D""></blockquote><br = class=3D"">That isn't master but old branch though. Could you run it = against<br class=3D"">*current* VDSM master?<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D"">On Mon, Apr 23, 2018 at = 3:56 PM, Martin Polednik <<a href=3D"mailto:mpolednik@redhat.com" = class=3D"">mpolednik@redhat.com</a>><br class=3D"">wrote:<br = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D"">On = 23/04/18 01:23 +0300, Elad Ben Aharon wrote:<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D"">Hi, I've triggered = another execution [1] due to some issues I saw in the<br class=3D"">first = which are not related to the patch.<br class=3D""><br class=3D"">The = success rate is 78% which is low comparing to tier1 executions with<br = class=3D"">code from downstream builds (95-100% success rates) [2].<br = class=3D""><br class=3D""></blockquote><br class=3D"">Could you run the = current master (without the dynamic_ownership patch)<br class=3D"">so = that we have viable comparision?<br class=3D""><br class=3D"">=46rom = what I could see so far, there is an issue with move and copy<br = class=3D""><blockquote type=3D"cite" class=3D"">operations to and from = Gluster domains. For example [3].<br class=3D""><br class=3D"">The logs = are attached.<br class=3D""><br class=3D""><br class=3D"">[1]<br = class=3D"">*<a = href=3D"https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv" = class=3D"">https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv</a>= <br class=3D"">-4.2-ge-runner-tier1-after-upgrade/7/testReport/<br = class=3D""><<a = href=3D"https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv" = class=3D"">https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv</a>= <br class=3D"">-4.2-ge-runner-tier1-after-upgrade/7/testReport/>*<br = class=3D""><br class=3D""><br class=3D""><br class=3D"">[2]<br = class=3D""><a = href=3D"https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/" = class=3D"">https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/</a><br= class=3D""><br class=3D"">rhv-4.2-ge-runner-tier1-after-upgrade/7/<br = class=3D""><br class=3D""><br class=3D""><br class=3D"">[3]<br = class=3D"">2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) = [vdsm.api] FINISH<br class=3D"">deleteImage error=3DImage does not exist = in domain:<br class=3D"">'image=3Dcabb8846-7a4b-4244-9835-5f603e682f33,<br= class=3D"">domain=3De5fd29c8-52ba-467e-be09-ca40ff054dd4'<br = class=3D"">from=3D:<br class=3D"">:ffff:10.35.161.182,40936, = flow_id=3Ddisks_syncAction_ba6b2630-5976-4935,<br = class=3D"">task_id=3D3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51)<br = class=3D"">2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) = [storage.TaskManager.Task]<br = class=3D"">(Task=3D'3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected = error (task:875)<br class=3D"">Traceback (most recent call last):<br = class=3D"">File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", = line 882,<br class=3D"">in<br class=3D"">_run<br class=3D""> = return fn(*args, **kargs)<br class=3D"">File "<string>", = line 2, in deleteImage<br class=3D"">File = "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in<br = class=3D"">method<br class=3D""> ret =3D func(*args, **kwargs)<br = class=3D"">File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", = line 1503,<br class=3D"">in<br class=3D"">deleteImage<br class=3D""> = raise se.ImageDoesNotExistInSD(imgUUID, sdUUID)<br = class=3D"">ImageDoesNotExistInSD: Image does not exist in domain:<br = class=3D"">'image=3Dcabb8846-7a4b-4244-9835-5f603e682f33,<br = class=3D"">domain=3De5fd29c8-52ba-467e-be09-ca40ff054dd4'<br = class=3D""><br class=3D"">2018-04-22 13:06:28,317+0300 INFO = (jsonrpc/7) [storage.TaskManager.Task]<br = class=3D"">(Task=3D'3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: = Task is aborted:<br class=3D"">"Image does not exist in domain: = 'image=3Dcabb8846-7a4b-4244-9835-<br class=3D"">5f603e682f33, = domain=3De5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268<br = class=3D"">(task:1181)<br class=3D"">2018-04-22 13:06:28,318+0300 ERROR = (jsonrpc/7) [storage.Dispatcher] FINISH<br class=3D"">deleteImage = error=3DImage does not exist in domain:<br = class=3D"">'image=3Dcabb8846-7a4b-4244-9835-5f603e682f33,<br = class=3D"">domain=3De5fd29c8-52ba-467e-be09<br class=3D"">-ca40ff054d<br = class=3D"">d4' (dispatcher:82)<br class=3D""><br class=3D""><br = class=3D""><br class=3D"">On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben = Aharon <ebenahar@redhat.com><br class=3D"">wrote:<br class=3D""><br = class=3D"">Triggered a sanity tier1 execution [1] using [2], which = covers all the<br class=3D""><blockquote type=3D"cite" = class=3D"">requested areas, on iSCSI, NFS and Gluster.<br class=3D"">I'll = update with the results.<br class=3D""><br class=3D"">[1]<br = class=3D"">https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2<br= class=3D"">_dev/job/rhv-4.2-ge-flow-storage/1161/<br class=3D""><br = class=3D"">[2]<br class=3D"">https://gerrit.ovirt.org/#/c/89830/<br = class=3D"">vdsm-4.30.0-291.git77aef9a.el7.x86_64<br class=3D""><br = class=3D""><br class=3D""><br class=3D"">On Thu, Apr 19, 2018 at 3:07 = PM, Martin Polednik <mpolednik@redhat.com><br class=3D"">wrote:<br = class=3D""><br class=3D"">On 19/04/18 14:54 +0300, Elad Ben Aharon = wrote:<br class=3D""><blockquote type=3D"cite" class=3D""><br = class=3D"">Hi Martin,<br class=3D""><blockquote type=3D"cite" = class=3D""><br class=3D"">I see [1] requires a rebase, can you please = take care?<br class=3D""><br class=3D""><br class=3D""></blockquote>Should= be rebased.<br class=3D""><br class=3D"">At the moment, our automation = is stable only on iSCSI, NFS, Gluster and<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D"">FC.<br class=3D"">Ceph = is not supported and Cinder will be stabilized soon, AFAIR, it's<br = class=3D"">not<br class=3D"">stable enough at the moment.<br = class=3D""><br class=3D""><br class=3D""></blockquote>That is still = pretty good.<br class=3D""><br class=3D""><br class=3D"">[1] = https://gerrit.ovirt.org/#/c/89830/<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D""><br class=3D""><br = class=3D"">Thanks<br class=3D""><br class=3D"">On Wed, Apr 18, 2018 at = 2:17 PM, Martin Polednik <mpolednik@redhat.com<br class=3D"">><br = class=3D"">wrote:<br class=3D""><br class=3D"">On 18/04/18 11:37 +0300, = Elad Ben Aharon wrote:<br class=3D""><br class=3D""><blockquote = type=3D"cite" class=3D""><br class=3D"">Hi, sorry if I misunderstood, I = waited for more input regarding what<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D"">areas<br class=3D"">have = to be tested here.<br class=3D""><br class=3D""><br class=3D"">I'd say = that you have quite a bit of freedom in this regard.<br = class=3D""></blockquote>GlusterFS<br class=3D"">should be covered by = Dennis, so iSCSI/NFS/ceph/cinder with some suite<br class=3D"">that = covers basic operations (start & stop VM, migrate it), snapshots<br = class=3D"">and merging them, and whatever else would be important for = storage<br class=3D"">sanity.<br class=3D""><br class=3D"">mpolednik<br = class=3D""><br class=3D""><br class=3D"">On Wed, Apr 18, 2018 at 11:16 = AM, Martin Polednik <<br class=3D"">mpolednik@redhat.com<br = class=3D"">><br class=3D""><br class=3D"">wrote:<br = class=3D""><blockquote type=3D"cite" class=3D""><br class=3D"">On = 11/04/18 16:52 +0300, Elad Ben Aharon wrote:<br class=3D""><br = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D"">We can = test this on iSCSI, NFS and GlusterFS. As for ceph and<br = class=3D"">cinder,<br class=3D""><br class=3D"">will<br = class=3D""><blockquote type=3D"cite" class=3D"">have to check, since = usually, we don't execute our automation on<br class=3D"">them.<br = class=3D""><br class=3D""><br class=3D"">Any update on this? I believe = the gluster tests were successful,<br class=3D"">OST<br class=3D""><br = class=3D""></blockquote>passes fine and unit tests pass fine, that makes = the storage<br class=3D"">backends<br class=3D"">test the last required = piece.<br class=3D""><br class=3D""><br class=3D"">On Wed, Apr 11, 2018 = at 4:38 PM, Raz Tamir <ratamir@redhat.com><br class=3D"">wrote:<br = class=3D""><br class=3D""><br class=3D"">+Elad<br class=3D""><blockquote = type=3D"cite" class=3D""><br class=3D""><br class=3D"">On Wed, Apr 11, = 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com<br = class=3D""><blockquote type=3D"cite" class=3D"">><br = class=3D"">wrote:<br class=3D""><br class=3D"">On Wed, Apr 11, 2018 at = 12:34 PM, Nir Soffer <nsoffer@redhat.com><br class=3D"">wrote:<br = class=3D""><br class=3D""><br class=3D"">On Wed, Apr 11, 2018 at 12:31 = PM Eyal Edri <eedri@redhat.com><br class=3D""><blockquote = type=3D"cite" class=3D"">wrote:<br class=3D""><br class=3D""><br = class=3D"">Please make sure to run as much OST suites on this patch = as<br class=3D""><blockquote type=3D"cite" class=3D"">possible<br = class=3D""><br class=3D"">before merging ( using 'ci please build' )<br = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D""><br = class=3D""><br class=3D"">But note that OST is not a way to verify the = patch.<br class=3D""><br class=3D""><br class=3D""></blockquote>Such = changes require testing with all storage types we support.<br = class=3D""><br class=3D"">Nir<br class=3D""><br class=3D"">On Tue, Apr = 10, 2018 at 4:09 PM, Martin Polednik <<br = class=3D"">mpolednik@redhat.com<br class=3D"">><br class=3D""><br = class=3D"">wrote:<br class=3D""><br class=3D""><br class=3D""><blockquote = type=3D"cite" class=3D"">Hey,<br class=3D""><br class=3D""><br = class=3D"">I've created a patch[0] that is finally able to activate<br = class=3D""><blockquote type=3D"cite" class=3D"">libvirt's<br = class=3D"">dynamic_ownership for VDSM while not negatively affecting<br = class=3D"">functionality of our storage code.<br class=3D""><br = class=3D"">That of course comes with quite a bit of code removal, = mostly<br class=3D"">in<br class=3D"">the<br class=3D"">area of host = devices, hwrng and anything that touches devices;<br class=3D"">bunch<br = class=3D"">of test changes and one XML generation caveat (storage is<br = class=3D"">handled<br class=3D"">by<br class=3D"">VDSM, therefore disk = relabelling needs to be disabled on the<br class=3D"">VDSM<br = class=3D"">level).<br class=3D""><br class=3D"">Because of the scope of = the patch, I welcome<br class=3D"">storage/virt/network<br = class=3D"">people to review the code and consider the implication = this<br class=3D"">change<br class=3D"">has<br class=3D"">on = current/future features.<br class=3D""><br class=3D"">[0] = https://gerrit.ovirt.org/#/c/89830/<br class=3D""><br class=3D""><br = class=3D"">In particular: dynamic_ownership was set to 0 = prehistorically<br class=3D"">(as<br class=3D""><br class=3D""><br = class=3D""></blockquote>part<br class=3D""><br class=3D""></blockquote><br= class=3D"">of https://bugzilla.redhat.com/show_bug.cgi?id=3D554961 ) = because<br class=3D""></blockquote>libvirt,<br class=3D"">running as = root, was not able to play properly with root-squash<br class=3D"">nfs<br = class=3D"">mounts.<br class=3D""><br class=3D"">Have you attempted this = use case?<br class=3D""><br class=3D"">I join to Nir's request to run = this with storage QE.<br class=3D""><br class=3D""><br class=3D""><br = class=3D""><br class=3D"">--<br class=3D""></blockquote><br class=3D""><br= class=3D"">Raz Tamir<br class=3D"">Manager, RHV QE<br class=3D""><br = class=3D""><br class=3D""><br class=3D""><br class=3D""><br = class=3D""></blockquote></blockquote></blockquote></blockquote></blockquot= e></blockquote></blockquote><br class=3D""></blockquote></blockquote><br = class=3D""><br = class=3D""></blockquote></blockquote>_____________________________________= __________<br class=3D"">Devel mailing list<br class=3D""><a = href=3D"mailto:Devel@ovirt.org" class=3D"">Devel@ovirt.org</a><br = class=3D"">http://lists.ovirt.org/mailman/listinfo/devel<br class=3D""><br= class=3D""><br class=3D""></div></div></blockquote></div><br = class=3D""></div></body></html>= --Apple-Mail=_807B5345-431A-48DA-A469-7819B1EAE0EF--
On 11/04/18 16:28 +0300, Dan Kenigsberg wrote:
On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com> wrote:
Please make sure to run as much OST suites on this patch as possible before merging ( using 'ci please build' )
But note that OST is not a way to verify the patch.
Such changes require testing with all storage types we support.
Nir
On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <mpolednik@redhat.com>
wrote:
Hey,
I've created a patch[0] that is finally able to activate libvirt's dynamic_ownership for VDSM while not negatively affecting functionality of our storage code.
That of course comes with quite a bit of code removal, mostly in the area of host devices, hwrng and anything that touches devices; bunch of test changes and one XML generation caveat (storage is handled by VDSM, therefore disk relabelling needs to be disabled on the VDSM level).
Because of the scope of the patch, I welcome storage/virt/network people to review the code and consider the implication this change has on current/future features.
In particular: dynamic_ownership was set to 0 prehistorically (as part of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because libvirt, running as root, was not able to play properly with root-squash nfs mounts.
Have you attempted this use case?
I have not. Added this to my to-do list. The important part to note about this patch (compared to my previous attempts in the past) is that it explicitly disables dynamic_ownership for FILE/BLOCK-backed disks. That means, unless `seclabel` is broken on libivrt side, the behavior would be unchanged for storage.
I join to Nir's request to run this with storage QE.
participants (8)
-
Dan Kenigsberg -
Elad Ben Aharon -
Eyal Edri -
Martin Polednik -
Michal Skrivanek -
Nir Soffer -
Raz Tamir -
Yaniv Kaul