April 2016 - Devel - oVirt List Archives

Integration tests future (and very nice alternative for the DAO fixture file)
by Roman Mohr 04 Jul '16

04 Jul '16

Hi all, In [1] you can find some patches which are meant to improve the test writing experience in ovirt-engine. They provide the following things: A) Domain Object builders which can be used for creating and/or persisting domain objects [2] B) DAO testing without writing fixtures because of the builders C) Integration testing for commands in conjunction with a real database Arquillian, injectable commands and the builders [3] # How to run what? A) In normal unit tests just create a new instance of a builder and use it. This should help us to get rid of all the small createDefaultVm(), createHostWithX() helper methods in our tests. B) In dao tests just inject them and go ahead. The advantage of not using the fixture file is that we can now set up clean scenarios for every test in a setup method. See example 2 below on how easy it is to set up a new cluster. C) Arquillian integration tests need to be marked with "@Category(IntegrationTest.class)" and can inherit from TransactionalTestBase. The @Category annotation makes sure that the integration tests are only run when mvn clean verify -DskipITs=false is invoked. Note that these tests are then executed in the integration test phase of maven. For them we use the maven-failsafe-plugin[5] which will also make sure that the testing database is up to date. See [4] for more details. # Examples 1) Add a running VM to a host, persist everything to the database and load all VMs which are running on the host: VDS host = vdsBuilder.cluster(persistedCluster).persist(); vmBuilder.host(host).up().persist(); List<VM> vms = vmDao.getAllRunningForVds(host.getId()); 2) Add 10 hosts with 1 GB of RAM to a cluster, persist the hosts to the database in a DAO test: public class MyHostDaoTest extends BaseDaoTestCase { @Inject private VdsBuilder vdsBuilder; @Test public void createHosts() { VdsBuilder builder = vdsBuilder.cluster(persistedCluster).physicalMemory(1000); for (int x =0; x < 10; x++){ builder.id(Guid.newGuid()).persist(); } } } 3) Full integration test with arquillian and the database @Category(IntegrationTest.class) public class VmDaoIntegrationTest extends TransactionalTestBase { @Inject VmDao vmDao; private final Guid VM1_GUID = Guid.createGuidFromString("0fe4bc81-5999-4ab6-80f8-7a4a2d4bfacd"); @Deployment public static JavaArchive deploy(){ return createDeployment(); } @Test public void shouldFailOnExistingEntity() { vmBuilder.id(VM1_GUID).cluster(clusterBuilder.reset().persist()).persist(); // This uses assertThat from assertj: assertThat(vmDao.get(VM1_GUID)).isNotNull(); } } 4) Using the builders in a normal unit test without a database: VM vm = new VmBuilder().id(Guid.newGuid()).up().build(); # How to add your own Domain objects? There are just a few simple rules: 1) Your builder should extend org.ovirt.engine.core.builder.AbstractBuilder 2) Make sure that you only access DAOs injected into the builder during #prePersist() and #persist(). This allows to use the #build() method also without injections 3) #prePersist() should set all fields which are necessary to suffice database constraints. The fields should only be set if they are not already set before by the builder. When following this rule we can always persist new objects to the database by simply calling myBuilder.reset().persist(). 4) Mark your builder with @Repository to make them useable for our Spring DAO tests and our Arquillian integration tests. So have a look at the patches at [1] and let me know what you think about them. Best Regards and happy testing, Roman [1] https://gerrit.ovirt.org/#/q/topic:integration [2] https://gerrit.ovirt.org/#/c/47008/17 [3] https://gerrit.ovirt.org/#/c/47007/10 [4] https://gerrit.ovirt.org/#/c/47008/17 [5] https://maven.apache.org/surefire/maven-failsafe-plugin/

5 7

Progress report on Vdsm Debian packaging
by Milan Zamazal 21 May '16

21 May '16

I made progress with Debian packaging of Vdsm. We have all the supporting packages (cpopen, ioprocess, mom, pthreading, safelease, sanlock) in Debian now. I took over maintenance of sanlock Debian package, updated it and made a lot of fixes, including preparing the previously unavailable python-sanlock package. As for Vdsm itself, the package can be installed now and Vdsm can be started. It doesn't work with Engine, which is no big surprise. We can't simply reinstall a Debian host from Engine, so we need to find some way how to initialize the host properly. And it's likely there are distribution specific pieces inside Vdsm and elsewhere. All the package sources, including not yet uploaded Vdsm package, are available on http://git.debian.org . How you can help: - Obviously, we need to make Vdsm hosts working, so any help in this area is welcome. - Reviewing and testing Vdsm Debian package (without actually using it) is also very helpful, I'm sure there are things to fix or improve. - There are some Lintian warnings in the packages, fixing them would be nice. - We should also review Debian patches in the packages, decide what should be changed upstream and report it. - You can help with anything else you consider useful.

4 6

[VDSM] flaky networking test
by Nir Soffer 01 May '16

01 May '16

Hi all, I think we have a pending patch for this, perhaps we should merge it? 19:32:54 ====================================================================== 19:32:54 ERROR: test_local_auto_with_dynamic_address_from_ra (network.netinfo_test.TestIPv6Addresses) 19:32:54 ---------------------------------------------------------------------- 19:32:54 Traceback (most recent call last): 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testValidation.py", line 97, in wrapper 19:32:54 return f(*args, **kwargs) 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/netinfo_test.py", line 446, in test_local_auto_with_dynamic_address_from_ra 19:32:54 self.assertEqual('link', ip_addrs[1]['scope']) 19:32:54 File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__ 19:32:54 self.gen.next() 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/nettestlib.py", line 405, in dnsmasq_run 19:32:54 server.stop() 19:32:54 File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__ 19:32:54 self.gen.next() 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/firewall.py", line 45, in allow_dhcp 19:32:54 _forbid_dhcp(iface) 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/firewall.py", line 89, in _forbid_dhcp 19:32:54 '67', '-j', 'ACCEPT']) # DHCPv4 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/firewall.py", line 110, in _execCmdChecker 19:32:54 command, out, err)) 19:32:54 FirewallError: Command ['/sbin/iptables', '-D', 'INPUT', '-i', 'veth_qjA94YJwY8', '-p', 'udp', '--sport', '68', '--dport', '67', '-j', 'ACCEPT'] failed with []; ['Another app is currently holding the xtables lock. Perhaps you want to use the -w option?'] 19:32:54 -------------------- >> begin captured logging << -------------------- 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip link add name veth_qjA94YJwY8 type veth peer name veth_NLC6ccMcUU (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /usr/sbin/dnsmasq --dhcp-authoritative -p 0 --dhcp-option=3 --dhcp-option=6 -i veth_qjA94YJwY8 -I lo -d --bind-dynamic --enable-ra --dhcp-range=2001:1:1:1::,slaac,2m (cwd None) 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/service iptables status (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = 'Redirecting to /bin/systemctl status iptables.service\nRunning in chroot, ignoring request.\n'; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/iptables --wait -I INPUT -i veth_qjA94YJwY8 -p udp --sport 68 --dport 67 -j ACCEPT (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/iptables --wait -I INPUT -i veth_qjA94YJwY8 -p udp --sport 546 --dport 547 -j ACCEPT (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip link set dev veth_NLC6ccMcUU up (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip link set dev veth_qjA94YJwY8 up (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip -6 addr add dev veth_qjA94YJwY8 2001:1:1:1::1/64 (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: dnsmasq: started, version 2.75 DNS disabled 19:32:54 dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify 19:32:54 dnsmasq: warning: interface veth_qjA94YJwY8 does not currently exist 19:32:54 dnsmasq-dhcp: router advertisement on 2001:1:1:1:: 19:32:54 dnsmasq-dhcp: IPv6 router advertisement enabled 19:32:54 dnsmasq-dhcp: RTR-SOLICIT(veth_qjA94YJwY8) ce:82:16:6e:a5:7d 19:32:54 dnsmasq-dhcp: RTR-ADVERT(veth_qjA94YJwY8) 2001:1:1:1:: 19:32:54 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/service iptables status (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = 'Redirecting to /bin/systemctl status iptables.service\nRunning in chroot, ignoring request.\n'; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/iptables -D INPUT -i veth_qjA94YJwY8 -p udp --sport 68 --dport 67 -j ACCEPT (cwd None) 19:32:55 root: DEBUG: FAILED: <err> = 'Another app is currently holding the xtables lock. Perhaps you want to use the -w option?\n'; <rc> = 4 19:32:55 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip link del dev veth_qjA94YJwY8 (cwd None) 19:32:55 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:55 --------------------- >> end captured logging << ---------------------

2 3

cannot create local ISO domain in RHEVM
by knarra 01 May '16

01 May '16

Hi, I am trying to set up Hosted Engine. I wanted to deploy windows virtual machines in this environment and as part of that i am creating ISO Domain to upload my windows image. But for some reason, while i create ISO domain it always fails with error "Failed to add Storage Domain <storagedomain_name"> and always remains in unattached state. Can some one please help me to reslove this issue? Attaching the vdsm logs. Thanks kasturi.

2 2

[REVIEW-REQUEST] Gluster Self-Heal Monitoring feature page
by Ramesh Nachimuthu 29 Apr '16

29 Apr '16

Hi, We are planning to implement "Gluster Self-Heal Monitoring" feature in oVirt 4.0. This is very important for host maintenance and host fencing flows in Ovirt-Gluster Hyperconvergence use cases. I have added a feature to discuss the feature in detail. Please review the same and provide your feedback. Feature Page: http://www.ovirt.org/develop/release-management/features/gluster-self-heal-… Regards, Ramesh

1 0

Vdsm api package
by Nir Soffer 29 Apr '16

29 Apr '16

Hi all, In the Vdsm call, we discussed a way to expose vdsm errors to its clients (e.g, engine, hosted engine agent/setup). The idea is to have a vdsmapi package, holding: - errors.py - all public errors - events.py - all events sent by vdsm - client.py - library for communicating with vdsm - schema.py - the client will use this to autogenerate online help and validate messages - schema.yaml - we probably need several files (gluster, events, etc.) This will allow other projects talking with vdsm to do: from vdsmapi import client, errors ... try: client.list(vmId="xxxyyy") except errors.NoSuchVM: ... (this is a fake example, the real api may be different) Engine can build-require vdsmapi, and generate java module with the public errors from vdsmapi/errors.py module, instead of keeping this hardcoded in engine, and updating it every time vdsm adds new public error. Vdsm will use this package when building response to clients. Edi was concerned about sharing the errors module, so maybe we need a package: vdsmapi/ errors/ network.py virt.py storage.py gluster.py We can still expose all the errors via errors/__init__.py, so clients do not have to care about the area of the application the error come from. Thoughts? Nir

5 7

[VDSM] Flaky stompTests tests
by Nir Soffer 29 Apr '16

29 Apr '16

Hi all, We see random failures in the stomp tests for long time. I suggest we mark these tests as broken for now. See examples at the end of the message. Nir ---- http://jenkins.phx.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/93/con… 17:53:09 ====================================================================== 17:53:09 ERROR: test_event(False) (stompTests.StompTests) 17:53:09 ---------------------------------------------------------------------- 17:53:09 Traceback (most recent call last): 17:53:09 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py", line 74, in wrapper 17:53:09 return f(self, *args) 17:53:09 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/stompTests.py", line 110, in test_event 17:53:09 client.callMethod("event", [], str(uuid4())) 17:53:09 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/__init__.py", line 340, in callMethod 17:53:09 raise JsonRpcNoResponseError(methodName) 17:53:09 JsonRpcNoResponseError: [-32605] No response for JSON-RPC event request. 17:53:09 -------------------- >> begin captured logging << -------------------- 17:53:09 vds.MultiProtocolAcceptor: INFO: Listening at ::1:36333 17:53:09 Scheduler: DEBUG: Starting scheduler test.Scheduler 17:53:09 Scheduler: DEBUG: started 17:53:09 Executor: DEBUG: Starting executor 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/0 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/1 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/2 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/3 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/4 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/5 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/6 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/7 17:53:09 Executor: DEBUG: Worker started 17:53:09 FakeClientIf: INFO: XMLRPC server running 17:53:09 vds.MultiProtocolAcceptor: DEBUG: Adding detector <vdsm.rpc.bindingxmlrpc.XmlDetector instance at 0x7f0191899488> 17:53:09 vds.MultiProtocolAcceptor: DEBUG: Adding detector <yajsonrpc.stompreactor.StompDetector instance at 0x7f01918994d0> 17:53:09 ProtocolDetector.AcceptorImpl: INFO: Accepting connection from ::1:39823 17:53:09 ProtocolDetector.Detector: DEBUG: Using required_size=11 17:53:09 ProtocolDetector.Detector: INFO: Detected protocol stomp from ::1:39823 17:53:09 jsonrpc.AsyncoreClient: DEBUG: Sending response 17:53:09 Broker.StompAdapter: INFO: Processing CONNECT request 17:53:09 yajsonrpc.protocols.stomp.AsyncClient: DEBUG: Stomp connection established 17:53:09 jsonrpc.JsonRpcServer: DEBUG: Calling 'event' in bridge with [] 17:53:09 jsonrpc.Notification: DEBUG: Sending event {"params": {"content": true, "notify_time": 6398709080}, "jsonrpc": "2.0", "method": "vdsm.event"} 17:53:09 jsonrpc.JsonRpcServer: DEBUG: Return 'event' in bridge with True 17:53:09 Broker.StompAdapter: INFO: Subscribe command received 17:53:09 protocoldetector.StompDetector: DEBUG: Stomp detected from ('::1', 39823) 17:53:09 vds.MultiProtocolAcceptor: DEBUG: Stopping Acceptor 17:53:09 jsonrpc.JsonRpcServer: INFO: Stopping JsonRPC Server 17:53:09 Executor: DEBUG: Stopping executor 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/0 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/7 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/2 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/1 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/5 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/3 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/6 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/4 17:53:09 FakeClientIf: INFO: Stopping XMLRPC server 17:53:09 FakeClientIf: INFO: XMLRPC server stopped 17:53:09 Scheduler: DEBUG: Stopping scheduler test.Scheduler 17:53:09 Scheduler: DEBUG: stopped 17:53:09 --------------------- >> end captured logging << --------------------- http://jenkins.phx.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/87/con… 17:14:24 ====================================================================== 17:14:24 ERROR: test_event(False) (stompTests.StompTests) 17:14:24 ---------------------------------------------------------------------- 17:14:24 Traceback (most recent call last): 17:14:24 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py", line 74, in wrapper 17:14:24 return f(self, *args) 17:14:24 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/stompTests.py", line 110, in test_event 17:14:24 client.callMethod("event", [], str(uuid4())) 17:14:24 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/__init__.py", line 340, in callMethod 17:14:24 raise JsonRpcNoResponseError(methodName) 17:14:24 JsonRpcNoResponseError: [-32605] No response for JSON-RPC event request. 17:14:24 -------------------- >> begin captured logging << -------------------- 17:14:24 vds.MultiProtocolAcceptor: INFO: Listening at ::1:39663 17:14:24 Scheduler: DEBUG: Starting scheduler test.Scheduler 17:14:24 Scheduler: DEBUG: started 17:14:24 Executor: DEBUG: Starting executor 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/0 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/1 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/2 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/3 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/4 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/5 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/6 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/7 17:14:24 Executor: DEBUG: Worker started 17:14:24 FakeClientIf: INFO: XMLRPC server running 17:14:24 vds.MultiProtocolAcceptor: DEBUG: Adding detector <vdsm.rpc.bindingxmlrpc.XmlDetector instance at 0x7fa3a183c6c8> 17:14:24 vds.MultiProtocolAcceptor: DEBUG: Adding detector <yajsonrpc.stompreactor.StompDetector instance at 0x7fa3a183c710> 17:14:24 ProtocolDetector.AcceptorImpl: INFO: Accepting connection from ::1:34641 17:14:24 ProtocolDetector.Detector: DEBUG: Using required_size=11 17:14:24 ProtocolDetector.Detector: INFO: Detected protocol stomp from ::1:34641 17:14:24 jsonrpc.AsyncoreClient: DEBUG: Sending response 17:14:24 Broker.StompAdapter: INFO: Processing CONNECT request 17:14:24 yajsonrpc.protocols.stomp.AsyncClient: DEBUG: Stomp connection established 17:14:24 jsonrpc.JsonRpcServer: DEBUG: Calling 'event' in bridge with [] 17:14:24 jsonrpc.Notification: DEBUG: Sending event {"params": {"content": true, "notify_time": 11313399420}, "jsonrpc": "2.0", "method": "vdsm.event"} 17:14:24 jsonrpc.JsonRpcServer: DEBUG: Return 'event' in bridge with True 17:14:24 Broker.StompAdapter: INFO: Subscribe command received 17:14:24 protocoldetector.StompDetector: DEBUG: Stomp detected from ('::1', 34641) 17:14:24 vds.MultiProtocolAcceptor: DEBUG: Stopping Acceptor 17:14:24 jsonrpc.JsonRpcServer: INFO: Stopping JsonRPC Server 17:14:24 Executor: DEBUG: Stopping executor 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/5 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/3 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/0 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/2 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/1 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/4 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/7 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/6 17:14:24 FakeClientIf: INFO: Stopping XMLRPC server 17:14:24 FakeClientIf: INFO: XMLRPC server stopped 17:14:24 Scheduler: DEBUG: Stopping scheduler test.Scheduler 17:14:24 Scheduler: DEBUG: stopped 17:14:24 --------------------- >> end captured logging << --------------------- http://jenkins.phx.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/82/con… 17:05:43 ====================================================================== 17:05:43 ERROR: test_event(False) (stompTests.StompTests) 17:05:43 ---------------------------------------------------------------------- 17:05:43 Traceback (most recent call last): 17:05:43 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py", line 74, in wrapper 17:05:43 return f(self, *args) 17:05:43 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/stompTests.py", line 110, in test_event 17:05:43 client.callMethod("event", [], str(uuid4())) 17:05:43 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/__init__.py", line 340, in callMethod 17:05:43 raise JsonRpcNoResponseError(methodName) 17:05:43 JsonRpcNoResponseError: [-32605] No response for JSON-RPC event request. 17:05:43 -------------------- >> begin captured logging << -------------------- 17:05:43 Scheduler: DEBUG: stopped 17:05:43 vds.MultiProtocolAcceptor: INFO: Listening at ::1:37245 17:05:43 Scheduler: DEBUG: Starting scheduler test.Scheduler 17:05:43 Scheduler: DEBUG: started 17:05:43 Executor: DEBUG: Starting executor 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/0 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/1 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/2 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/3 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/4 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/5 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/6 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/7 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Worker started 17:05:43 FakeClientIf: INFO: XMLRPC server running 17:05:43 vds.MultiProtocolAcceptor: DEBUG: Adding detector <vdsm.rpc.bindingxmlrpc.XmlDetector instance at 0x7f27fb0d70e0> 17:05:43 vds.MultiProtocolAcceptor: DEBUG: Adding detector <yajsonrpc.stompreactor.StompDetector instance at 0x7f27fb0e2488> 17:05:43 ProtocolDetector.AcceptorImpl: INFO: Accepting connection from ::1:52346 17:05:43 ProtocolDetector.Detector: DEBUG: Using required_size=11 17:05:43 jsonrpc.AsyncoreClient: DEBUG: Sending response 17:05:43 ProtocolDetector.Detector: INFO: Detected protocol stomp from ::1:52346 17:05:43 Broker.StompAdapter: INFO: Processing CONNECT request 17:05:43 yajsonrpc.protocols.stomp.AsyncClient: DEBUG: Stomp connection established 17:05:43 jsonrpc.JsonRpcServer: DEBUG: Calling 'event' in bridge with [] 17:05:43 jsonrpc.Notification: DEBUG: Sending event {"params": {"content": true, "notify_time": 6397426400}, "jsonrpc": "2.0", "method": "vdsm.event"} 17:05:43 jsonrpc.JsonRpcServer: DEBUG: Return 'event' in bridge with True 17:05:43 Broker.StompAdapter: INFO: Subscribe command received 17:05:43 protocoldetector.StompDetector: DEBUG: Stomp detected from ('::1', 52346) 17:05:43 vds.MultiProtocolAcceptor: DEBUG: Stopping Acceptor 17:05:43 jsonrpc.JsonRpcServer: INFO: Stopping JsonRPC Server 17:05:43 Executor: DEBUG: Stopping executor 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/6 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/3 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/1 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/4 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/0 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/5 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/2 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/7 17:05:43 FakeClientIf: INFO: Stopping XMLRPC server 17:05:43 FakeClientIf: INFO: XMLRPC server stopped 17:05:43 Scheduler: DEBUG: Stopping scheduler test.Scheduler 17:05:43 Scheduler: DEBUG: stopped 17:05:43 --------------------- >> end captured logging << --------------------- http://jenkins.phx.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/72/con… 16:32:00 ====================================================================== 16:32:00 ERROR: test_echo(4096, False) (stompTests.StompTests) 16:32:00 ---------------------------------------------------------------------- 16:32:00 Traceback (most recent call last): 16:32:00 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py", line 74, in wrapper 16:32:00 return f(self, *args) 16:32:00 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/stompTests.py", line 90, in test_echo 16:32:00 str(uuid4())), 16:32:00 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/__init__.py", line 340, in callMethod 16:32:00 raise JsonRpcNoResponseError(methodName) 16:32:00 JsonRpcNoResponseError: [-32605] No response for JSON-RPC echo request. 16:32:00 -------------------- >> begin captured logging << -------------------- 16:32:00 vds.MultiProtocolAcceptor: INFO: Listening at ::1:59025 16:32:00 Scheduler: DEBUG: Starting scheduler test.Scheduler 16:32:00 Scheduler: DEBUG: started 16:32:00 Executor: DEBUG: Starting executor 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/0 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/1 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/2 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/3 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/4 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/5 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/6 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/7 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Worker started 16:32:00 FakeClientIf: INFO: XMLRPC server running 16:32:00 vds.MultiProtocolAcceptor: DEBUG: Adding detector <vdsm.rpc.bindingxmlrpc.XmlDetector instance at 0x7f411903ecf8> 16:32:00 vds.MultiProtocolAcceptor: DEBUG: Adding detector <yajsonrpc.stompreactor.StompDetector instance at 0x7f411903ed40> 16:32:00 ProtocolDetector.AcceptorImpl: INFO: Accepting connection from ::1:49920 16:32:00 ProtocolDetector.Detector: DEBUG: Using required_size=11 16:32:00 ProtocolDetector.Detector: INFO: Detected protocol stomp from ::1:49920 16:32:00 Broker.StompAdapter: INFO: Processing CONNECT request 16:32:00 jsonrpc.AsyncoreClient: DEBUG: Sending response 16:32:00 Broker.StompAdapter: INFO: Subscribe command received 16:32:00 vds.dispatcher: ERROR: uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher ('::1', 49920, 0, 0) at 0x7f411903ef38> (<type 'exceptions.AttributeError'>:'NoneType' object has no attribute 'command' [/usr/lib64/python2.7/asyncore.py|readwrite|108] [/usr/lib64/python2.7/asyncore.py|handle_read_event|449] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|63] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|137] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/vdsm/protocoldetector.py|handle_read|125] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|handle_socket|481] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|add_socket|48] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|createListener|449] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|StompListener|415] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|__init__|39] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|switch_implementation|72] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|init|433] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|_onAccept|51] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|set_message_handler|297] [/usr/lib64/python2.7/asyncore.py|handle_read_event|449] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|63] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|137] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stomp.py|handle_read|367] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|handle_frame|260]) 16:32:00 vds.dispatcher: WARNING: unhandled close event 16:32:00 jsonrpc.JsonRpcServer: DEBUG: Calling 'echo' in bridge with [u'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod '] 16:32:01 jsonrpc.JsonRpcServer: DEBUG: Return 'echo' in bridge with Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod 16:32:01 yajsonrpc.protocols.stomp.AsyncClient: DEBUG: Stomp connection established 16:32:01 vds.MultiProtocolAcceptor: DEBUG: Stopping Acceptor 16:32:01 jsonrpc.JsonRpcServer: INFO: Stopping JsonRPC Server 16:32:01 Executor: DEBUG: Stopping executor 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/5 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/6 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/4 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/0 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/2 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/1 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/7 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/3 16:32:01 FakeClientIf: INFO: Stopping XMLRPC server 16:32:01 FakeClientIf: INFO: XMLRPC server stopped 16:32:01 Scheduler: DEBUG: Stopping scheduler test.Scheduler 16:32:01 Scheduler: DEBUG: stopped 16:32:01 --------------------- >> end captured logging << ---------------------

4 4

Pull request for gluster self-heal monitoring feature page https://github.com/oVirt/ovirt-site/pull/214
by Ramesh Nachimuthu 29 Apr '16

29 Apr '16

Hi, I have a pull request for "Gluster Slef Heal Monitoring" feature page. Can someone help me with the review?. Regards, Rames

2 1

A question about vdsm
by zhukaijie 28 Apr '16

28 Apr '16

[vm_id].recovery file is generated which /var/run/vdsm/, which contains some VM config info that I want to prevent from other users. So are there any configurations or ways I can use to disable generation of the file? thank you.

3 2

[ACTION REQUIRED][ATTENTION NEEDED] move gluster repo to CentOS Storage SIG
by Sandro Bonazzola 28 Apr '16

28 Apr '16

Hi, this week we faced an outage on download.gluster.org server which broke our CI and caused users to be unable to install oVirt on centOS during the outage due to missing gluster packages. We've been advised by gluster community to use CentOS Storage SIG gluster rpms instead of community ones. I've started moving our release rpm to point to it starting from master ( https://gerrit.ovirt.org/56751 ) and I'll slowly go over our projects. In the meanwhile, if you've standard-ci based automation and you require gluster repositories, please move to CentOS Storage SIG as well. Repository is at http://mirror.centos.org/centos/$releasever/storage/$basearch/gluster-3.7/ <http://mirror.centos.org/centos/$releasever/storage/$basearch/gluster-3.7/> Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

1 0