[ovirt-devel] [RFC] Release process

Mon Sep 15 13:56:33 UTC 2014

Adding infra mailing lists to the loop.

Il 08/09/2014 11:34, Sandro Bonazzola ha scritto:
> Il 20/08/2014 10:05, Sandro Bonazzola ha scritto:
>> Hi,
>> while we're near to release 3.5.0 I would like to start the discussion on the release process for 3.5.z and 3.6.0.
>> The reference page for the release process has been last updated 2 years ago and is not reflecting the way we're managing the releases anymore.
> 
> Following up to this email with proposal for missing bits. Comments are welcome.
> 
> 
> 
>> In order to start the discussion here's what we are currently doing:
>>
>> = oVirt maintenance releases =
>>
>> * After a new 3.y.0 release, maintenance releases are scheduled every month until a new 3.y+1.0 release is published.
>> * After first 3.y.0 release candidate is published a new release management wiki page is created (see http://www.ovirt.org/OVirt_3.4.z_release-management)
>> * For each 3.y.z release, the release management page is updated
>> ** Defining the schedule for RC and GA date
>> ** Pointing to the new release note page to be created (see http://www.ovirt.org/OVirt_3.4.4_Release_Notes)
>> ** Pointing to a testing page to be created (see http://www.ovirt.org/Testing/oVirt_3.4.4_Testing)
>> ** Pointing to a blocker bug to be created (see http://bugzilla.redhat.com/1118689)
>> * Weekly status email are sent to users and devel lists
>> * QA page is updated with pointers to the new release (see http://www.ovirt.org/OVirt_Quality_Assurance)
>> * Release manager update the release notes pages starting from RC
>>
>> Usually RC date is scheduled one week before GA.
>> RC releases are created by taking the latest nightly snapshot available after verifying that the build passes basic sanity test and repository closure.
>> A new stabilization branch is created with the same git hash used for building the snapshot.
>> Between RC and GA release criteria are tested on the RC and if the release met the criteria a GA release will be built just updating the versioning
>> code to drop master, git hash and timestamp suffix from tarballs and rpms.
>> A new RC will be created and GA postponed by one week if release criteria are not met.
>>
>>
>>
>> = oVirt Release Process =
>>
>> * After a new 3.y.0 release, a new 3.y+1.0 release is tentatively scheduled after 6 months.
>> * A new release management page is created (see http://www.ovirt.org/OVirt_3.5_release-management)
>> * A new tracker bug is created (see http://bugzilla.redhat.com/1073943)
>> * A discussion is started on devel and users mailing list gathering idea for next release features
>> * Teams prepare a list of accepted features collecting / creating bug tracker, devel owner, qa owner, feature page for each of them
>> * Several presentations are scheduled by the teams presenting the accepted features
>> * An alpha release is scheduled 4 months before GA
>> * Feature freeze is scheduled 3 months before GA
>> * A beta release is scheduled 2 months before GA, git tree is branched for stabilization
>> * A release candidate is scheduled 1 month before GA
>> * Weekly status email are sent starting 3 weeks before alpha release.
>> * Test days are scheduled after every milestone release
>> * Release manager update the release notes pages starting from Alpha
>>
>> All milestones releases are created by taking the latest nightly snapshot available after verifying that the build passes basic sanity test and
>> repository closure.
>>
>> RC will be postponed until all known blockers are fixed
>>
>> Between RC and GA release criteria are tested on the RC and if the release met the criteria a GA release will be built just updating the versioning
>> code to drop master, git hash and timestamp suffix from tarballs and rpms.
>> A new RC will be created and GA postponed by one week if release criteria are not met.
>>
>> [[Category:Release management]]
>>
>>
>> What I think we're missing / doing wrong:
>> * A discussion about release criteria *after* accepted features list is ready
> 
> 1) I think we should require that each feature must come with a set of test cases covering the functionality provided by the new feature.
> The test cases becomes part of the release criteria as
> " * All test cases defined for each accepted feature must be verified and pass the test for the build to be released"
> 
> This means that the whole test suite must be verified on each release candidate for deciding if it's ready to be released.
> Test suite should cover whatever is in the documentation of the feature.
> This means that the feature must contain enough documentation to be considered "a contract" between the feature owner and the feature tester / user.
> 
> 
> 2) I think that the current criteria: "MUST: No blockers on the lower level components - libvirt, lvm,device-mapper,qemu-kvm, Jboss, postgres,
> iscsi-initiator" should be dropped.
> We're not acting as QA for lower level components. Instead I think we should change above statement with:
> "MUST: all lower level components must be available on supported distributions at required version"
> 
> This means we're not hosting any package we don't develop inside oVirt repository just because it's not yet released officially.
> 
> I think we can allow to require third party repositories like JPackage, Gluster, virt-preview, epel-testing, fedora-updates-testing.
> We shouldn't allow to add to oVirt repositories packages which belongs to third party repositories.
> Example of those are libguestfs, python-cpopen, slf4j, kexec-tools and so on.
> 
> 
> 3) MUST: No known regressions from previous releases.
> this means that bugs with Regression keyword will become automatically blockers.
> 
> 4) MUST: Have Release Notes with feature specific information
> I think this should be a requirement for entering beta phase: all features page should contains enough information for allowing features maintainers
> to add some information to the release notes page.
> 
> 5) MUST: Have release announcement for front page of ovirt.org and for mailing lists
> Well, this is something usually we do after we released the packages, so it can't be a MUST.
> 
> 6) MUST: All sources must be available on ovirt.org
> 
> I think all remaining existing requirements should be on a per subproject base.
> 
> 7) MUST each subproject must allow upgrade from a previous version either providing documented manual or automated upgrade procedures.
> 
> 8) MUST each subproject must pass it's own regression test matrix.
> This means that each subproject maintainer should create one and ensure it's tested.
> This can be rephrased as "All features working on previous release must still work in the new release if not dropped as deprecated"
> While verifying the features, regression bugs can be found. We just can't re-verify all fixed bugs, can we?
> 
> 9) MUST each subproject should list packages expected to be in the release.
> This just for ensuring we can verify the final repository for completeness.
> 
> 
> Points 1 and 8 ensures also we have a test day page ready for the first test day, just listing test cases to be covered.
> 
>> * A clear schedule like http://fedoraproject.org/wiki/Releases/21/Schedule
> 
> Will follow up on the schedule once intermediate steps are more defined.
> 
> 
>> * A clear feature freeze policy like https://fedoraproject.org/wiki/Changes_Freeze_Policy
> 
> Ignoring the Feature Freeze
> Introducing significant changes after Feature Freeze can result in your patch to be reverted.
> 
> Allowed after Feature Freeze:
> - Adding code meant for automated testing of the feature
> - Adding bug fixes
> - Adding localization
> - Cleaning or optimizing the code without introducing new features
> 
> Not allowed after Feature Freeze:
> - Continue to add new enhancements (note that supporting a new distribution is an enhancement)
> - Try to cover new enhancements under bug fixes
> - Make changes that require other subprojects to make changes as well (API, ABI and configuration files included) if not absolutely required for
> fixing a bug
> - Make changes that require packages not yet available on supported distributions.
> 
> Exceptions
> About this, I think we shouldn't allow exceptions.
> But if needed, I think that a public vote on the exception should be required.
> Exceptions should be asked at least one week before feature freeze and vote on them should be closed on feature freeze.
> 
> This means that you're not allowed to open new features but you can complete those that can't be completed for feature freeze if the vote allow it.
> 
> 
> Incomplete features
> I think we should require for each feature to define a contingency plan, allowing either to disable the incomplete feature or ensuring that the
> feature code is isolated in a directory or package that can be easily dropped.
> Those that can't be done that way may live in their own branch until they're finished and merged in the main tree just when completed.
> 
> 
>> * A clear policy about changes we must allow and not allow between RC and GA
> 
> I think that only patches fixing blockers bug should be allowed between RC and GA.
> This means that ystream branches must fork on RC release git hash and release maintainers must ensure nothing will be merged there if not related to a
> blocker bug.
> 
>> * Alpha release should come after feature freeze
> 
> Nothing more to say about it
> 
>> * A string freeze schedule, allowing translators to have enough time to translate new strings
> 
> I think that we should freeze string between beta and 1 week before RC.
> Considering we have 1 month between beta and RC this means that string freeze should go no later than 2 weeks before RC and translators have 2 weeks
> for finalizing last translation bits and verify them.
> So, I think we should have a beta release requirement:
> - Supported localizations must be at least at 70% of completeness at beta stage for being included in the release.
> Without such requirement translators won't make it on time for GA.
> 
> 
>> * Maintainers should pay more attention to release notes and test day pages
> 
> This is enforced by rules on features acceptance requirements.
> 
>>
>>
> 
> 

-- 
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com