Hi all, I tested today [RFE] replace XML-RPC communication (engine-vdsm) with json-rpc based on bidirectional transport First I upgraded ovirt-3.4 stable engine to ovirt-3.5 - ok Then I upgraded 4 hosts to latest vdsm - ok I upgraded 2 data centers to cluster version 3.5: - 2 Fedora 19 hosts with 30 ISCSI storage domains - ok - 2 RHEL 6.5 hosts with 45 NFS storage domains - failed I had to remove the hosts and the virtual machines to complete the upgrade [1] Then I removed the hosts and added them back (to configure jsonrpc), and setup one host using jsonrpc and the other using xmlrpc - ok After moving the hosts to maintenance mode and starting them back, I found that the host using jsonrpc was stuck in "Unassigned" state [2],[3]. The errors in the vdsm log were not clear enough. After I improving this [4], I could fix it in one line patch [5]. Finally when I had a working system, I run some sanity tests: - start/stop vm - ok - create vm from template - ok - migrate vms between two hosts concurrenly (one host use xmlrpc, one using json) - ok Then I tried to test create template from vm, but I had low disk space on that storage domain. So I tried to extend the domain which would be useful test as well. But turns out that you cannot create or edit a block domain when using jsonrpc [6] Looking at the logs, I found also that shutting down protocol detector fails [7] Summary: - upgrade is broken in some cases - critical - jsonrpc is not ready yet - jsonrpc needs lot of additional testing - for next test day I suggest one tester from each team (virt, storage, networking, sla?) to test jsonrpc with relevant flows. [1] https://bugzilla.redhat.com/1114994 Cannot edit cluster after upgrade from version 3.4 to 3.5 because cpu type (Intel Haswell) does not match [2] https://bugzilla.redhat.com/1115033 StoragePool_disconnect: disconnect() takes exactly 4 arguments [3] https://bugzilla.redhat.com/1115044 Host stuck in "Unassinged" state when using jsonrpc and disconnection from pool failed [4] http://gerrit.ovirt.org/29457 bridge: Show more info when method call fail [5] http://gerrit.ovirt.org/29465 api: Make remove optional [6] https://bugzilla.redhat.com/show_bug.cgi?id=1115152 Cannot edit or create block storage doamin when using jsonrpc [7] https://bugzilla.redhat.com/1115104 Shuting down protocol detector fails Nir
nice test, Nir. would you mind adding the important bugs to 3.5 tracker [1]? I'd agree that not being able to upgrade without removal of all hosts and vms is a blocker for a release:-) others seem to be quite important too Thanks, michal [1] http://bugzilla.redhat.com/1073943 On Jul 1, 2014, at 20:31 , Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
I tested today [RFE] replace XML-RPC communication (engine-vdsm) with json-rpc based on bidirectional transport
First I upgraded ovirt-3.4 stable engine to ovirt-3.5 - ok Then I upgraded 4 hosts to latest vdsm - ok
I upgraded 2 data centers to cluster version 3.5: - 2 Fedora 19 hosts with 30 ISCSI storage domains - ok - 2 RHEL 6.5 hosts with 45 NFS storage domains - failed I had to remove the hosts and the virtual machines to complete the upgrade [1]
Then I removed the hosts and added them back (to configure jsonrpc), and setup one host using jsonrpc and the other using xmlrpc - ok
After moving the hosts to maintenance mode and starting them back, I found that the host using jsonrpc was stuck in "Unassigned" state [2],[3].
The errors in the vdsm log were not clear enough. After I improving this [4], I could fix it in one line patch [5].
Finally when I had a working system, I run some sanity tests: - start/stop vm - ok - create vm from template - ok - migrate vms between two hosts concurrenly (one host use xmlrpc, one using json) - ok
Then I tried to test create template from vm, but I had low disk space on that storage domain. So I tried to extend the domain which would be useful test as well.
But turns out that you cannot create or edit a block domain when using jsonrpc [6]
Looking at the logs, I found also that shutting down protocol detector fails [7]
Summary:
- upgrade is broken in some cases - critical - jsonrpc is not ready yet - jsonrpc needs lot of additional testing - for next test day I suggest one tester from each team (virt, storage, networking, sla?) to test jsonrpc with relevant flows.
[1] https://bugzilla.redhat.com/1114994 Cannot edit cluster after upgrade from version 3.4 to 3.5 because cpu type (Intel Haswell) does not match
[2] https://bugzilla.redhat.com/1115033 StoragePool_disconnect: disconnect() takes exactly 4 arguments
[3] https://bugzilla.redhat.com/1115044 Host stuck in "Unassinged" state when using jsonrpc and disconnection from pool failed
[4] http://gerrit.ovirt.org/29457 bridge: Show more info when method call fail
[5] http://gerrit.ovirt.org/29465 api: Make remove optional
[6] https://bugzilla.redhat.com/show_bug.cgi?id=1115152 Cannot edit or create block storage doamin when using jsonrpc
[7] https://bugzilla.redhat.com/1115104 Shuting down protocol detector fails
Nir _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (2)
-
Michal Skrivanek -
Nir Soffer