Move bugs to ON_QA based on snapshot builds?
Hello everyone, I wanted to suggest that we automate the change of oVirt 3.6 bug status from MODIFIED to ON_QA based on nightly snapshot builds. The motivation should be clear: when bugs are fixed on the master branch, the fix becomes readily available for verification as soon as the next snapshot is built (and there's indeed someone to verify on the master branch - the same person who was working on the master branch and opened the bug!). We currently only move them to ON_QA on milestone builds (e.g. alpha, beta), but I don't think that's right for an open source project - the status of bugs (targeted to the nearest release) should be up-to-date with the actual state of the master branch. We've encountered the problem testing features for 3.6 the last couple of weeks, and since it's gonna be a long version this situation will likely occur often. So far I've been moving bugs to ON_QA myself, but I don't really want to follow the snapshot builds manually (nor move the bugs to ON_QA as soon as they're merged, in case the snapshot build doesn't happen). The downside would be that bugs could be VERIFIED at an early point in the development, and later regressions could occur that would render the verification obsolete. But this could happen just the same between release milestones... Thoughts? Yours, Lior.
----- Original Message -----
From: "Lior Vernia" <lvernia@redhat.com> To: devel@ovirt.org, infra@ovirt.org Sent: Wednesday, January 28, 2015 10:44:44 AM Subject: Move bugs to ON_QA based on snapshot builds?
Hello everyone,
I wanted to suggest that we automate the change of oVirt 3.6 bug status from MODIFIED to ON_QA based on nightly snapshot builds.
The motivation should be clear: when bugs are fixed on the master branch, the fix becomes readily available for verification as soon as the next snapshot is built (and there's indeed someone to verify on the master branch - the same person who was working on the master branch and opened the bug!).
We currently only move them to ON_QA on milestone builds (e.g. alpha, beta), but I don't think that's right for an open source project - the status of bugs (targeted to the nearest release) should be up-to-date with the actual state of the master branch.
We've encountered the problem testing features for 3.6 the last couple of weeks, and since it's gonna be a long version this situation will likely occur often. So far I've been moving bugs to ON_QA myself, but I don't really want to follow the snapshot builds manually (nor move the bugs to ON_QA as soon as they're merged, in case the snapshot build doesn't happen).
The downside would be that bugs could be VERIFIED at an early point in the development, and later regressions could occur that would render the verification obsolete. But this could happen just the same between release milestones...
Thoughts?
I'm all for this. By the time we reach Alpha, we'll have several hundred bugs in MODIFIED - if all of these are dumped to ON_QA simultaneously it will be impossible to manage/prioritize. +1.
Yours, Lior. _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Il 28/01/2015 10:44, Lior Vernia ha scritto:
Hello everyone,
I wanted to suggest that we automate the change of oVirt 3.6 bug status from MODIFIED to ON_QA based on nightly snapshot builds.
The motivation should be clear: when bugs are fixed on the master branch, the fix becomes readily available for verification as soon as the next snapshot is built (and there's indeed someone to verify on the master branch - the same person who was working on the master branch and opened the bug!).
We currently only move them to ON_QA on milestone builds (e.g. alpha, beta), but I don't think that's right for an open source project - the status of bugs (targeted to the nearest release) should be up-to-date with the actual state of the master branch.
We've encountered the problem testing features for 3.6 the last couple of weeks, and since it's gonna be a long version this situation will likely occur often. So far I've been moving bugs to ON_QA myself, but I don't really want to follow the snapshot builds manually (nor move the bugs to ON_QA as soon as they're merged, in case the snapshot build doesn't happen).
The downside would be that bugs could be VERIFIED at an early point in the development, and later regressions could occur that would render the verification obsolete. But this could happen just the same between release milestones...
Thoughts?
+1 I agree, once the RPM including the fix is available in master snapshot it's ready for QE testing.
Yours, Lior.
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
Hi, just a quick question: Would this imply that you need to create separate bugs to track these "solved on master" bugs for the next release? So you know the fix/feature really hit's the release? Or how would you track that? -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
On 28/01/15 11:53, Sven Kieske wrote:
Hi,
just a quick question:
Would this imply that you need to create separate bugs to track these "solved on master" bugs for the next release?
So you know the fix/feature really hit's the release?
Or how would you track that?
I would stop the automated moving to ON_QA from snapshot builds as soon as we create a stable ovirt-3.6 branch. At that point you know that any existing fix will really hit the release, and onwards only patches backported to the stable branch will be moved to ON_QA as stable branch releases are available (whether nightly, weekly, monthly). So no need for separate bugs.
participants (4)
-
Allon Mureinik -
Lior Vernia -
Sandro Bonazzola -
Sven Kieske