- Devel - oVirt List Archives

Reminder: oVirt 4 Feature freeze and beta release in 2 weeks
by Sandro Bonazzola 02 May '16

02 May '16

Hi, just a friendly reminder that we're entering feature freeze in 2 weeks and release oVirt 4.0 beta on May 18th[1]. If your project is not yet built in jenkins and / or not yet published in master repository please let me know. [1] http://www.ovirt.org/develop/release-management/releases/4.0/release-manage… -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

1 0

Weekly bug status for ovirt-system-tests (using Lago) [ 01/05/16 ]
by Eyal Edri 01 May '16

01 May '16

FYI, Some updates from recent activity on ovirt-system-tests running on Lago. *Open bugs:* https://goo.gl/eUaPUY *Open issues:* Improvements on the host install issue, some env are not reproducing anymore but not sure we're 100% clear yet. *Open patches:* (reviews are welcome) https://gerrit.ovirt.org/#/q/project:ovirt-system-tests+status:open * There are quite a few open, some are just temp for helping with the debug of 'stuck in installing state' *Jenkins jobs: (failing on known issues)* http://jenkins.ovirt.org/view/oVirt%20system%20tests/ *Recently merged patches:* commit 211e475c004a2635896e528cd3aed6cd7d0dc5fd Author: Sandro Bonazzola <sbonazzo(a)redhat.com> Date: Thu Apr 28 16:44:50 2016 +0200 common: yum-repos: update glusterfs repos - dropped Gluster community repo for Fedora23: the repo is empty and Gluster community is maintaining Fedora packages within Fedora repositories. For reference: http://download.gluster.org/pub/gluster/glusterfs/LATEST/Fedora/What_About_… - on EL7 moved from Gluster community repo to CentOS Storage SIG repositories: we've been advised by Gluster community that their packaging effort is focused on CentOS Storage SIG and CentOS infrastructure is more reliable. Change-Id: I4d2c752be825fbbb9035222845728d10c5dd288c Signed-off-by: Sandro Bonazzola <sbonazzo(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit 19c102110279975d274c32477cc28c7c29e75fbc Author: David Caro <dcaroest(a)redhat.com> Date: Wed Apr 20 21:10:37 2016 +0200 Fixed answerfile for master A new question was added to the setup Change-Id: I6be958a70d67126d8067ff1855018125623b10c1 ---------------------------------------------------------------------------------------------------------- commit 5829862d17e92f91cfd04e7b9d5549cd775c800c Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Mon Apr 25 14:06:27 2016 +0300 Add 'device=eth0' to EL7 deploy scripts. Apparently, /etc/sysconfig/network-scripts/ifcfg-eth0 lacks it on EL7 images. This works with NetworkManager, but not without it. When something goes bad, VDSM fails to rollback properly as its missing. For now, add it. Change-Id: I1c4ca07499a3947022e6f943cf05842ee1def408 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit ddb42782968e5c6d411cdc747503fd90bed249d7 Author: David Caro <dcaroest(a)redhat.com> Date: Tue Mar 29 18:53:42 2016 +0200 Fix the master suite runs This is just copying over and adapting a bit (just so it works) the 3.6 suite into the master suite. Only for the 3.6 api, the newapi tests will be added later Still affected by the storage deactivation issues Change-Id: I55781a5d5a746aaf81e403eeeee40880beacb8a0 Signed-off-by: David Caro <dcaroest(a)redhat.com> ---------------------------------------------------------------------------------------------------------- commit c7d50cf8aebd6b566ae5c6ee8d6e3b8ec24ffc2b Author: Yaniv Kaul <ykaul(a)redhat.com> Date: Tue Apr 26 13:57:43 2016 +0300 Remove snapshot and reverting as part of the ovirt-system-tests I currently did not find a use case for it. It slows down the tests and causes (real? false?) issues with SDs. Mainly their activation and deactivation. It dramatically reduced the time to run the whole suite for me. 3-4 minutes on my desktop, and >10 minutes on my laptop. Change-Id: I9f7e509220c83805df78abb94b0ddc390c4a7f54 Signed-off-by: Yaniv Kaul <ykaul(a)redhat.com> ---------------------------------------------------------------------------------------------------------- -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0

[VDSM] flaky networking test
by Nir Soffer 01 May '16

01 May '16

Hi all, I think we have a pending patch for this, perhaps we should merge it? 19:32:54 ====================================================================== 19:32:54 ERROR: test_local_auto_with_dynamic_address_from_ra (network.netinfo_test.TestIPv6Addresses) 19:32:54 ---------------------------------------------------------------------- 19:32:54 Traceback (most recent call last): 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testValidation.py", line 97, in wrapper 19:32:54 return f(*args, **kwargs) 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/netinfo_test.py", line 446, in test_local_auto_with_dynamic_address_from_ra 19:32:54 self.assertEqual('link', ip_addrs[1]['scope']) 19:32:54 File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__ 19:32:54 self.gen.next() 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/nettestlib.py", line 405, in dnsmasq_run 19:32:54 server.stop() 19:32:54 File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__ 19:32:54 self.gen.next() 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/firewall.py", line 45, in allow_dhcp 19:32:54 _forbid_dhcp(iface) 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/firewall.py", line 89, in _forbid_dhcp 19:32:54 '67', '-j', 'ACCEPT']) # DHCPv4 19:32:54 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/network/firewall.py", line 110, in _execCmdChecker 19:32:54 command, out, err)) 19:32:54 FirewallError: Command ['/sbin/iptables', '-D', 'INPUT', '-i', 'veth_qjA94YJwY8', '-p', 'udp', '--sport', '68', '--dport', '67', '-j', 'ACCEPT'] failed with []; ['Another app is currently holding the xtables lock. Perhaps you want to use the -w option?'] 19:32:54 -------------------- >> begin captured logging << -------------------- 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip link add name veth_qjA94YJwY8 type veth peer name veth_NLC6ccMcUU (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /usr/sbin/dnsmasq --dhcp-authoritative -p 0 --dhcp-option=3 --dhcp-option=6 -i veth_qjA94YJwY8 -I lo -d --bind-dynamic --enable-ra --dhcp-range=2001:1:1:1::,slaac,2m (cwd None) 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/service iptables status (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = 'Redirecting to /bin/systemctl status iptables.service\nRunning in chroot, ignoring request.\n'; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/iptables --wait -I INPUT -i veth_qjA94YJwY8 -p udp --sport 68 --dport 67 -j ACCEPT (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/iptables --wait -I INPUT -i veth_qjA94YJwY8 -p udp --sport 546 --dport 547 -j ACCEPT (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip link set dev veth_NLC6ccMcUU up (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip link set dev veth_qjA94YJwY8 up (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip -6 addr add dev veth_qjA94YJwY8 2001:1:1:1::1/64 (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:54 root: DEBUG: dnsmasq: started, version 2.75 DNS disabled 19:32:54 dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify 19:32:54 dnsmasq: warning: interface veth_qjA94YJwY8 does not currently exist 19:32:54 dnsmasq-dhcp: router advertisement on 2001:1:1:1:: 19:32:54 dnsmasq-dhcp: IPv6 router advertisement enabled 19:32:54 dnsmasq-dhcp: RTR-SOLICIT(veth_qjA94YJwY8) ce:82:16:6e:a5:7d 19:32:54 dnsmasq-dhcp: RTR-ADVERT(veth_qjA94YJwY8) 2001:1:1:1:: 19:32:54 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/service iptables status (cwd None) 19:32:54 root: DEBUG: SUCCESS: <err> = 'Redirecting to /bin/systemctl status iptables.service\nRunning in chroot, ignoring request.\n'; <rc> = 0 19:32:54 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/iptables -D INPUT -i veth_qjA94YJwY8 -p udp --sport 68 --dport 67 -j ACCEPT (cwd None) 19:32:55 root: DEBUG: FAILED: <err> = 'Another app is currently holding the xtables lock. Perhaps you want to use the -w option?\n'; <rc> = 4 19:32:55 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 /sbin/ip link del dev veth_qjA94YJwY8 (cwd None) 19:32:55 root: DEBUG: SUCCESS: <err> = ''; <rc> = 0 19:32:55 --------------------- >> end captured logging << ---------------------

2 3

cannot create local ISO domain in RHEVM
by knarra 01 May '16

01 May '16

Hi, I am trying to set up Hosted Engine. I wanted to deploy windows virtual machines in this environment and as part of that i am creating ISO Domain to upload my windows image. But for some reason, while i create ISO domain it always fails with error "Failed to add Storage Domain <storagedomain_name"> and always remains in unattached state. Can some one please help me to reslove this issue? Attaching the vdsm logs. Thanks kasturi.

2 2

[REVIEW-REQUEST] Gluster Self-Heal Monitoring feature page
by Ramesh Nachimuthu 29 Apr '16

29 Apr '16

Hi, We are planning to implement "Gluster Self-Heal Monitoring" feature in oVirt 4.0. This is very important for host maintenance and host fencing flows in Ovirt-Gluster Hyperconvergence use cases. I have added a feature to discuss the feature in detail. Please review the same and provide your feedback. Feature Page: http://www.ovirt.org/develop/release-management/features/gluster-self-heal-… Regards, Ramesh

1 0

Vdsm api package
by Nir Soffer 29 Apr '16

29 Apr '16

Hi all, In the Vdsm call, we discussed a way to expose vdsm errors to its clients (e.g, engine, hosted engine agent/setup). The idea is to have a vdsmapi package, holding: - errors.py - all public errors - events.py - all events sent by vdsm - client.py - library for communicating with vdsm - schema.py - the client will use this to autogenerate online help and validate messages - schema.yaml - we probably need several files (gluster, events, etc.) This will allow other projects talking with vdsm to do: from vdsmapi import client, errors ... try: client.list(vmId="xxxyyy") except errors.NoSuchVM: ... (this is a fake example, the real api may be different) Engine can build-require vdsmapi, and generate java module with the public errors from vdsmapi/errors.py module, instead of keeping this hardcoded in engine, and updating it every time vdsm adds new public error. Vdsm will use this package when building response to clients. Edi was concerned about sharing the errors module, so maybe we need a package: vdsmapi/ errors/ network.py virt.py storage.py gluster.py We can still expose all the errors via errors/__init__.py, so clients do not have to care about the area of the application the error come from. Thoughts? Nir

5 7

[VDSM] Flaky stompTests tests
by Nir Soffer 29 Apr '16

29 Apr '16

Hi all, We see random failures in the stomp tests for long time. I suggest we mark these tests as broken for now. See examples at the end of the message. Nir ---- http://jenkins.phx.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/93/con… 17:53:09 ====================================================================== 17:53:09 ERROR: test_event(False) (stompTests.StompTests) 17:53:09 ---------------------------------------------------------------------- 17:53:09 Traceback (most recent call last): 17:53:09 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py", line 74, in wrapper 17:53:09 return f(self, *args) 17:53:09 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/stompTests.py", line 110, in test_event 17:53:09 client.callMethod("event", [], str(uuid4())) 17:53:09 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/__init__.py", line 340, in callMethod 17:53:09 raise JsonRpcNoResponseError(methodName) 17:53:09 JsonRpcNoResponseError: [-32605] No response for JSON-RPC event request. 17:53:09 -------------------- >> begin captured logging << -------------------- 17:53:09 vds.MultiProtocolAcceptor: INFO: Listening at ::1:36333 17:53:09 Scheduler: DEBUG: Starting scheduler test.Scheduler 17:53:09 Scheduler: DEBUG: started 17:53:09 Executor: DEBUG: Starting executor 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/0 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/1 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/2 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/3 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/4 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/5 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/6 17:53:09 Executor: DEBUG: Worker started 17:53:09 Executor: DEBUG: Starting worker jsonrpc.Executor/7 17:53:09 Executor: DEBUG: Worker started 17:53:09 FakeClientIf: INFO: XMLRPC server running 17:53:09 vds.MultiProtocolAcceptor: DEBUG: Adding detector <vdsm.rpc.bindingxmlrpc.XmlDetector instance at 0x7f0191899488> 17:53:09 vds.MultiProtocolAcceptor: DEBUG: Adding detector <yajsonrpc.stompreactor.StompDetector instance at 0x7f01918994d0> 17:53:09 ProtocolDetector.AcceptorImpl: INFO: Accepting connection from ::1:39823 17:53:09 ProtocolDetector.Detector: DEBUG: Using required_size=11 17:53:09 ProtocolDetector.Detector: INFO: Detected protocol stomp from ::1:39823 17:53:09 jsonrpc.AsyncoreClient: DEBUG: Sending response 17:53:09 Broker.StompAdapter: INFO: Processing CONNECT request 17:53:09 yajsonrpc.protocols.stomp.AsyncClient: DEBUG: Stomp connection established 17:53:09 jsonrpc.JsonRpcServer: DEBUG: Calling 'event' in bridge with [] 17:53:09 jsonrpc.Notification: DEBUG: Sending event {"params": {"content": true, "notify_time": 6398709080}, "jsonrpc": "2.0", "method": "vdsm.event"} 17:53:09 jsonrpc.JsonRpcServer: DEBUG: Return 'event' in bridge with True 17:53:09 Broker.StompAdapter: INFO: Subscribe command received 17:53:09 protocoldetector.StompDetector: DEBUG: Stomp detected from ('::1', 39823) 17:53:09 vds.MultiProtocolAcceptor: DEBUG: Stopping Acceptor 17:53:09 jsonrpc.JsonRpcServer: INFO: Stopping JsonRPC Server 17:53:09 Executor: DEBUG: Stopping executor 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Worker stopped 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/0 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/7 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/2 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/1 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/5 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/3 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/6 17:53:09 Executor: DEBUG: Waiting for worker jsonrpc.Executor/4 17:53:09 FakeClientIf: INFO: Stopping XMLRPC server 17:53:09 FakeClientIf: INFO: XMLRPC server stopped 17:53:09 Scheduler: DEBUG: Stopping scheduler test.Scheduler 17:53:09 Scheduler: DEBUG: stopped 17:53:09 --------------------- >> end captured logging << --------------------- http://jenkins.phx.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/87/con… 17:14:24 ====================================================================== 17:14:24 ERROR: test_event(False) (stompTests.StompTests) 17:14:24 ---------------------------------------------------------------------- 17:14:24 Traceback (most recent call last): 17:14:24 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py", line 74, in wrapper 17:14:24 return f(self, *args) 17:14:24 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/stompTests.py", line 110, in test_event 17:14:24 client.callMethod("event", [], str(uuid4())) 17:14:24 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/__init__.py", line 340, in callMethod 17:14:24 raise JsonRpcNoResponseError(methodName) 17:14:24 JsonRpcNoResponseError: [-32605] No response for JSON-RPC event request. 17:14:24 -------------------- >> begin captured logging << -------------------- 17:14:24 vds.MultiProtocolAcceptor: INFO: Listening at ::1:39663 17:14:24 Scheduler: DEBUG: Starting scheduler test.Scheduler 17:14:24 Scheduler: DEBUG: started 17:14:24 Executor: DEBUG: Starting executor 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/0 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/1 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/2 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/3 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/4 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/5 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/6 17:14:24 Executor: DEBUG: Worker started 17:14:24 Executor: DEBUG: Starting worker jsonrpc.Executor/7 17:14:24 Executor: DEBUG: Worker started 17:14:24 FakeClientIf: INFO: XMLRPC server running 17:14:24 vds.MultiProtocolAcceptor: DEBUG: Adding detector <vdsm.rpc.bindingxmlrpc.XmlDetector instance at 0x7fa3a183c6c8> 17:14:24 vds.MultiProtocolAcceptor: DEBUG: Adding detector <yajsonrpc.stompreactor.StompDetector instance at 0x7fa3a183c710> 17:14:24 ProtocolDetector.AcceptorImpl: INFO: Accepting connection from ::1:34641 17:14:24 ProtocolDetector.Detector: DEBUG: Using required_size=11 17:14:24 ProtocolDetector.Detector: INFO: Detected protocol stomp from ::1:34641 17:14:24 jsonrpc.AsyncoreClient: DEBUG: Sending response 17:14:24 Broker.StompAdapter: INFO: Processing CONNECT request 17:14:24 yajsonrpc.protocols.stomp.AsyncClient: DEBUG: Stomp connection established 17:14:24 jsonrpc.JsonRpcServer: DEBUG: Calling 'event' in bridge with [] 17:14:24 jsonrpc.Notification: DEBUG: Sending event {"params": {"content": true, "notify_time": 11313399420}, "jsonrpc": "2.0", "method": "vdsm.event"} 17:14:24 jsonrpc.JsonRpcServer: DEBUG: Return 'event' in bridge with True 17:14:24 Broker.StompAdapter: INFO: Subscribe command received 17:14:24 protocoldetector.StompDetector: DEBUG: Stomp detected from ('::1', 34641) 17:14:24 vds.MultiProtocolAcceptor: DEBUG: Stopping Acceptor 17:14:24 jsonrpc.JsonRpcServer: INFO: Stopping JsonRPC Server 17:14:24 Executor: DEBUG: Stopping executor 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Worker stopped 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/5 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/3 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/0 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/2 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/1 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/4 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/7 17:14:24 Executor: DEBUG: Waiting for worker jsonrpc.Executor/6 17:14:24 FakeClientIf: INFO: Stopping XMLRPC server 17:14:24 FakeClientIf: INFO: XMLRPC server stopped 17:14:24 Scheduler: DEBUG: Stopping scheduler test.Scheduler 17:14:24 Scheduler: DEBUG: stopped 17:14:24 --------------------- >> end captured logging << --------------------- http://jenkins.phx.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/82/con… 17:05:43 ====================================================================== 17:05:43 ERROR: test_event(False) (stompTests.StompTests) 17:05:43 ---------------------------------------------------------------------- 17:05:43 Traceback (most recent call last): 17:05:43 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py", line 74, in wrapper 17:05:43 return f(self, *args) 17:05:43 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/stompTests.py", line 110, in test_event 17:05:43 client.callMethod("event", [], str(uuid4())) 17:05:43 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/__init__.py", line 340, in callMethod 17:05:43 raise JsonRpcNoResponseError(methodName) 17:05:43 JsonRpcNoResponseError: [-32605] No response for JSON-RPC event request. 17:05:43 -------------------- >> begin captured logging << -------------------- 17:05:43 Scheduler: DEBUG: stopped 17:05:43 vds.MultiProtocolAcceptor: INFO: Listening at ::1:37245 17:05:43 Scheduler: DEBUG: Starting scheduler test.Scheduler 17:05:43 Scheduler: DEBUG: started 17:05:43 Executor: DEBUG: Starting executor 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/0 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/1 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/2 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/3 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/4 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/5 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/6 17:05:43 Executor: DEBUG: Starting worker jsonrpc.Executor/7 17:05:43 Executor: DEBUG: Worker started 17:05:43 Executor: DEBUG: Worker started 17:05:43 FakeClientIf: INFO: XMLRPC server running 17:05:43 vds.MultiProtocolAcceptor: DEBUG: Adding detector <vdsm.rpc.bindingxmlrpc.XmlDetector instance at 0x7f27fb0d70e0> 17:05:43 vds.MultiProtocolAcceptor: DEBUG: Adding detector <yajsonrpc.stompreactor.StompDetector instance at 0x7f27fb0e2488> 17:05:43 ProtocolDetector.AcceptorImpl: INFO: Accepting connection from ::1:52346 17:05:43 ProtocolDetector.Detector: DEBUG: Using required_size=11 17:05:43 jsonrpc.AsyncoreClient: DEBUG: Sending response 17:05:43 ProtocolDetector.Detector: INFO: Detected protocol stomp from ::1:52346 17:05:43 Broker.StompAdapter: INFO: Processing CONNECT request 17:05:43 yajsonrpc.protocols.stomp.AsyncClient: DEBUG: Stomp connection established 17:05:43 jsonrpc.JsonRpcServer: DEBUG: Calling 'event' in bridge with [] 17:05:43 jsonrpc.Notification: DEBUG: Sending event {"params": {"content": true, "notify_time": 6397426400}, "jsonrpc": "2.0", "method": "vdsm.event"} 17:05:43 jsonrpc.JsonRpcServer: DEBUG: Return 'event' in bridge with True 17:05:43 Broker.StompAdapter: INFO: Subscribe command received 17:05:43 protocoldetector.StompDetector: DEBUG: Stomp detected from ('::1', 52346) 17:05:43 vds.MultiProtocolAcceptor: DEBUG: Stopping Acceptor 17:05:43 jsonrpc.JsonRpcServer: INFO: Stopping JsonRPC Server 17:05:43 Executor: DEBUG: Stopping executor 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/6 17:05:43 Executor: DEBUG: Worker stopped 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/3 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/1 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/4 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/0 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/5 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/2 17:05:43 Executor: DEBUG: Waiting for worker jsonrpc.Executor/7 17:05:43 FakeClientIf: INFO: Stopping XMLRPC server 17:05:43 FakeClientIf: INFO: XMLRPC server stopped 17:05:43 Scheduler: DEBUG: Stopping scheduler test.Scheduler 17:05:43 Scheduler: DEBUG: stopped 17:05:43 --------------------- >> end captured logging << --------------------- http://jenkins.phx.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/72/con… 16:32:00 ====================================================================== 16:32:00 ERROR: test_echo(4096, False) (stompTests.StompTests) 16:32:00 ---------------------------------------------------------------------- 16:32:00 Traceback (most recent call last): 16:32:00 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testlib.py", line 74, in wrapper 16:32:00 return f(self, *args) 16:32:00 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/stompTests.py", line 90, in test_echo 16:32:00 str(uuid4())), 16:32:00 File "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/__init__.py", line 340, in callMethod 16:32:00 raise JsonRpcNoResponseError(methodName) 16:32:00 JsonRpcNoResponseError: [-32605] No response for JSON-RPC echo request. 16:32:00 -------------------- >> begin captured logging << -------------------- 16:32:00 vds.MultiProtocolAcceptor: INFO: Listening at ::1:59025 16:32:00 Scheduler: DEBUG: Starting scheduler test.Scheduler 16:32:00 Scheduler: DEBUG: started 16:32:00 Executor: DEBUG: Starting executor 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/0 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/1 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/2 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/3 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/4 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/5 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/6 16:32:00 Executor: DEBUG: Starting worker jsonrpc.Executor/7 16:32:00 Executor: DEBUG: Worker started 16:32:00 Executor: DEBUG: Worker started 16:32:00 FakeClientIf: INFO: XMLRPC server running 16:32:00 vds.MultiProtocolAcceptor: DEBUG: Adding detector <vdsm.rpc.bindingxmlrpc.XmlDetector instance at 0x7f411903ecf8> 16:32:00 vds.MultiProtocolAcceptor: DEBUG: Adding detector <yajsonrpc.stompreactor.StompDetector instance at 0x7f411903ed40> 16:32:00 ProtocolDetector.AcceptorImpl: INFO: Accepting connection from ::1:49920 16:32:00 ProtocolDetector.Detector: DEBUG: Using required_size=11 16:32:00 ProtocolDetector.Detector: INFO: Detected protocol stomp from ::1:49920 16:32:00 Broker.StompAdapter: INFO: Processing CONNECT request 16:32:00 jsonrpc.AsyncoreClient: DEBUG: Sending response 16:32:00 Broker.StompAdapter: INFO: Subscribe command received 16:32:00 vds.dispatcher: ERROR: uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher ('::1', 49920, 0, 0) at 0x7f411903ef38> (<type 'exceptions.AttributeError'>:'NoneType' object has no attribute 'command' [/usr/lib64/python2.7/asyncore.py|readwrite|108] [/usr/lib64/python2.7/asyncore.py|handle_read_event|449] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|63] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|137] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/vdsm/protocoldetector.py|handle_read|125] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|handle_socket|481] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|add_socket|48] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|createListener|449] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|StompListener|415] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|__init__|39] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|switch_implementation|72] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|init|433] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|_onAccept|51] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|set_message_handler|297] [/usr/lib64/python2.7/asyncore.py|handle_read_event|449] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|63] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|137] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stomp.py|handle_read|367] [/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/lib/yajsonrpc/stompreactor.py|handle_frame|260]) 16:32:00 vds.dispatcher: WARNING: unhandled close event 16:32:00 jsonrpc.JsonRpcServer: DEBUG: Calling 'echo' in bridge with [u'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod '] 16:32:01 jsonrpc.JsonRpcServer: DEBUG: Return 'echo' in bridge with Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod 16:32:01 yajsonrpc.protocols.stomp.AsyncClient: DEBUG: Stomp connection established 16:32:01 vds.MultiProtocolAcceptor: DEBUG: Stopping Acceptor 16:32:01 jsonrpc.JsonRpcServer: INFO: Stopping JsonRPC Server 16:32:01 Executor: DEBUG: Stopping executor 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Worker stopped 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/5 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/6 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/4 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/0 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/2 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/1 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/7 16:32:01 Executor: DEBUG: Waiting for worker jsonrpc.Executor/3 16:32:01 FakeClientIf: INFO: Stopping XMLRPC server 16:32:01 FakeClientIf: INFO: XMLRPC server stopped 16:32:01 Scheduler: DEBUG: Stopping scheduler test.Scheduler 16:32:01 Scheduler: DEBUG: stopped 16:32:01 --------------------- >> end captured logging << ---------------------

4 4

Pull request for gluster self-heal monitoring feature page https://github.com/oVirt/ovirt-site/pull/214
by Ramesh Nachimuthu 29 Apr '16

29 Apr '16

Hi, I have a pull request for "Gluster Slef Heal Monitoring" feature page. Can someone help me with the review?. Regards, Rames

2 1

A question about vdsm
by zhukaijie 28 Apr '16

28 Apr '16

[vm_id].recovery file is generated which /var/run/vdsm/, which contains some VM config info that I want to prevent from other users. So are there any configurations or ways I can use to disable generation of the file? thank you.

3 2

[ACTION REQUIRED][ATTENTION NEEDED] move gluster repo to CentOS Storage SIG
by Sandro Bonazzola 28 Apr '16

28 Apr '16

Hi, this week we faced an outage on download.gluster.org server which broke our CI and caused users to be unable to install oVirt on centOS during the outage due to missing gluster packages. We've been advised by gluster community to use CentOS Storage SIG gluster rpms instead of community ones. I've started moving our release rpm to point to it starting from master ( https://gerrit.ovirt.org/56751 ) and I'll slowly go over our projects. In the meanwhile, if you've standard-ci based automation and you require gluster repositories, please move to CentOS Storage SIG as well. Repository is at http://mirror.centos.org/centos/$releasever/storage/$basearch/gluster-3.7/ <http://mirror.centos.org/centos/$releasever/storage/$basearch/gluster-3.7/> Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

1 0