July 2018 - Devel - oVirt List Archives

[VDSM] test_echo(1024, False) (stomp_test.StompTests) fails
by Nir Soffer 01 Dec '18

01 Dec '18

This tests used to fail in the past, but since we fixed it or the code it never failed. Maybe the slave was overloaded? *14:19:02* ======================================================================*14:19:02* ERROR: *14:19:02* ----------------------------------------------------------------------*14:19:02* Traceback (most recent call last):*14:19:02* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/testlib.py", line 143, in wrapper*14:19:02* return f(self, *args)*14:19:02* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/stomp_test.py", line 95, in test_echo*14:19:02* str(uuid4())),*14:19:02* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/yajsonrpc/jsonrpcclient.py", line 77, in callMethod*14:19:02* raise exception.JsonRpcNoResponseError(method=methodName)*14:19:02* JsonRpcNoResponseError: No response for JSON-RPC request: {'method': 'echo'}*14:19:02* -------------------- >> begin captured logging << --------------------*14:19:02* 2018-06-27 14:17:54,524 DEBUG (MainThread) [vds.MultiProtocolAcceptor] Creating socket (host='::1', port=0, family=10, socketype=1, proto=6) (protocoldetector:225)*14:19:02* 2018-06-27 14:17:54,526 INFO (MainThread) [vds.MultiProtocolAcceptor] Listening at ::1:36713 (protocoldetector:183)*14:19:02* 2018-06-27 14:17:54,535 DEBUG (MainThread) [Scheduler] Starting scheduler test.Scheduler (schedule:98)*14:19:02* 2018-06-27 14:17:54,537 DEBUG (test.Scheduler) [Scheduler] START thread <Thread(test.Scheduler, started daemon 140562082535168)> (func=<bound method Scheduler._run of <vdsm.schedule.Scheduler object at 0x7fd74ca00390>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,538 DEBUG (test.Scheduler) [Scheduler] started (schedule:140)*14:19:02* 2018-06-27 14:17:54,546 DEBUG (JsonRpc (StompReactor)) [root] START thread <Thread(JsonRpc (StompReactor), started daemon 140562629256960)> (func=<bound method StompReactor.process_requests of <yajsonrpc.stompserver.StompReactor object at 0x7fd74ca006d0>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,547 DEBUG (MainThread) [Executor] Starting executor (executor:128)*14:19:02* 2018-06-27 14:17:54,549 DEBUG (MainThread) [Executor] Starting worker jsonrpc/0 (executor:286)*14:19:02* 2018-06-27 14:17:54,553 DEBUG (jsonrpc/0) [Executor] START thread <Thread(jsonrpc/0, started daemon 140562612471552)> (func=<bound method _Worker._run of <Worker name=jsonrpc/0 waiting task#=0 at 0x7fd74ca00650>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,554 DEBUG (jsonrpc/0) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,557 DEBUG (MainThread) [Executor] Starting worker jsonrpc/1 (executor:286)*14:19:02* 2018-06-27 14:17:54,558 DEBUG (MainThread) [Executor] Starting worker jsonrpc/2 (executor:286)*14:19:02* 2018-06-27 14:17:54,559 DEBUG (jsonrpc/2) [Executor] START thread <Thread(jsonrpc/2, started daemon 140562620864256)> (func=<bound method _Worker._run of <Worker name=jsonrpc/2 waiting task#=0 at 0x7fd74c9fb350>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,560 DEBUG (jsonrpc/2) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,561 DEBUG (MainThread) [Executor] Starting worker jsonrpc/3 (executor:286)*14:19:02* 2018-06-27 14:17:54,562 DEBUG (jsonrpc/3) [Executor] START thread <Thread(jsonrpc/3, started daemon 140562124498688)> (func=<bound method _Worker._run of <Worker name=jsonrpc/3 waiting task#=0 at 0x7fd74bc80290>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,563 DEBUG (jsonrpc/3) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,564 DEBUG (MainThread) [Executor] Starting worker jsonrpc/4 (executor:286)*14:19:02* 2018-06-27 14:17:54,565 DEBUG (jsonrpc/4) [Executor] START thread <Thread(jsonrpc/4, started daemon 140562116105984)> (func=<bound method _Worker._run of <Worker name=jsonrpc/4 waiting task#=0 at 0x7fd74bc80790>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,566 DEBUG (jsonrpc/4) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,568 DEBUG (jsonrpc/1) [Executor] START thread <Thread(jsonrpc/1, started daemon 140562132891392)> (func=<bound method _Worker._run of <Worker name=jsonrpc/1 waiting task#=0 at 0x7fd74c9fb690>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,569 DEBUG (jsonrpc/1) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,569 DEBUG (MainThread) [Executor] Starting worker jsonrpc/5 (executor:286)*14:19:02* 2018-06-27 14:17:54,570 DEBUG (jsonrpc/5) [Executor] START thread <Thread(jsonrpc/5, started daemon 140562107713280)> (func=<bound method _Worker._run of <Worker name=jsonrpc/5 waiting task#=0 at 0x7fd74bc80090>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,571 DEBUG (jsonrpc/5) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,572 DEBUG (MainThread) [Executor] Starting worker jsonrpc/6 (executor:286)*14:19:02* 2018-06-27 14:17:54,580 DEBUG (MainThread) [Executor] Starting worker jsonrpc/7 (executor:286)*14:19:02* 2018-06-27 14:17:54,603 DEBUG (jsonrpc/6) [Executor] START thread <Thread(jsonrpc/6, started daemon 140562099320576)> (func=<bound method _Worker._run of <Worker name=jsonrpc/6 waiting task#=0 at 0x7fd74d0dacd0>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,604 DEBUG (jsonrpc/6) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,615 DEBUG (jsonrpc/7) [Executor] START thread <Thread(jsonrpc/7, started daemon 140562090927872)> (func=<bound method _Worker._run of <Worker name=jsonrpc/7 waiting task#=0 at 0x7fd74bfe0310>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,616 DEBUG (JsonRpcServer) [root] START thread <Thread(JsonRpcServer, started daemon 140561730238208)> (func=<bound method JsonRpcServer.serve_requests of <yajsonrpc.JsonRpcServer object at 0x7fd74ca00690>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,618 DEBUG (MainThread) [vds.MultiProtocolAcceptor] Adding detector <yajsonrpc.stompserver.StompDetector instance at 0x7fd74c28cc68> (protocoldetector:210)*14:19:02* 2018-06-27 14:17:54,625 INFO (Detector thread) [ProtocolDetector.AcceptorImpl] Accepted connection from ::1:52432 (protocoldetector:61)*14:19:02* 2018-06-27 14:17:54,626 DEBUG (Detector thread) [ProtocolDetector.Detector] Using required_size=11 (protocoldetector:89)*14:19:02* 2018-06-27 14:17:54,627 DEBUG (jsonrpc/7) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,634 DEBUG (MainThread) [jsonrpc.AsyncoreClient] Sending response (stompclient:293)*14:19:02* 2018-06-27 14:17:54,634 DEBUG (Client ::1:36713) [root] START thread <Thread(Client ::1:36713, started daemon 140561713452800)> (func=<bound method Reactor.process_requests of <yajsonrpc.betterAsyncore.Reactor object at 0x7fd74bdb1d90>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,641 INFO (Detector thread) [ProtocolDetector.Detector] Detected protocol stomp from ::1:52432 (protocoldetector:125)*14:19:02* 2018-06-27 14:17:54,645 INFO (Detector thread) [Broker.StompAdapter] Processing CONNECT request (stompserver:94)*14:19:02* 2018-06-27 14:17:54,648 DEBUG (Client ::1:36713) [yajsonrpc.protocols.stomp.AsyncClient] Stomp connection established (stompclient:137)*14:19:02* 2018-06-27 14:17:54,651 DEBUG (jsonrpc/0) [jsonrpc.JsonRpcServer] Calling 'echo' in bridge with [u'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut en'] (__init__:328)*14:19:02* 2018-06-27 14:17:54,652 DEBUG (jsonrpc/0) [jsonrpc.JsonRpcServer] Return 'echo' in bridge with Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut en (__init__:355)*14:19:02* 2018-06-27 14:17:54,653 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call echo succeeded in 0.01 seconds (__init__:311)*14:19:02* 2018-06-27 14:17:54,661 INFO (Detector thread) [Broker.StompAdapter] Subscribe command received (stompserver:123)*14:19:02* 2018-06-27 14:17:54,662 DEBUG (Detector thread) [protocoldetector.StompDetector] Stomp detected from ('::1', 52432) (stompserver:412)*14:19:02* 2018-06-27 14:18:09,649 DEBUG (MainThread) [vds.MultiProtocolAcceptor] Stopping Acceptor (protocoldetector:214)*14:19:02* 2018-06-27 14:18:09,650 INFO (MainThread) [jsonrpc.JsonRpcServer] Stopping JsonRPC Server (__init__:441)*14:19:02* 2018-06-27 14:18:09,651 DEBUG (MainThread) [Executor] Stopping executor (executor:137)*14:19:02* 2018-06-27 14:18:09,652 DEBUG (Client ::1:36713) [root] FINISH thread <Thread(Client ::1:36713, started daemon 140561713452800)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,652 DEBUG (JsonRpcServer) [root] FINISH thread <Thread(JsonRpcServer, started daemon 140561730238208)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,654 DEBUG (JsonRpc (StompReactor)) [root] FINISH thread <Thread(JsonRpc (StompReactor), started daemon 140562629256960)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,655 DEBUG (jsonrpc/2) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,658 DEBUG (jsonrpc/3) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,659 DEBUG (jsonrpc/4) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,660 DEBUG (jsonrpc/1) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,660 DEBUG (jsonrpc/5) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,660 DEBUG (jsonrpc/6) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,665 DEBUG (jsonrpc/6) [Executor] FINISH thread <Thread(jsonrpc/6, started daemon 140562099320576)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,661 DEBUG (jsonrpc/7) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,662 DEBUG (jsonrpc/2) [Executor] FINISH thread <Thread(jsonrpc/2, started daemon 140562620864256)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,663 DEBUG (jsonrpc/3) [Executor] FINISH thread <Thread(jsonrpc/3, started daemon 140562124498688)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,663 DEBUG (jsonrpc/1) [Executor] FINISH thread <Thread(jsonrpc/1, started daemon 140562132891392)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,664 DEBUG (jsonrpc/4) [Executor] FINISH thread <Thread(jsonrpc/4, started daemon 140562116105984)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,664 DEBUG (jsonrpc/0) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,665 DEBUG (jsonrpc/5) [Executor] FINISH thread <Thread(jsonrpc/5, started daemon 140562107713280)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,667 DEBUG (jsonrpc/7) [Executor] FINISH thread <Thread(jsonrpc/7, started daemon 140562090927872)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,667 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/0 (executor:290)*14:19:02* 2018-06-27 14:18:09,671 DEBUG (jsonrpc/0) [Executor] FINISH thread <Thread(jsonrpc/0, started daemon 140562612471552)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,673 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/1 (executor:290)*14:19:02* 2018-06-27 14:18:09,674 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/6 (executor:290)*14:19:02* 2018-Coverage.py warning: Module /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm was never imported. (module-not-imported)*14:19:05* pylint installdeps: pylint==1.8*14:19:22* pylint installed: The directory '/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.,astroid==1.6.5,backports.functools-lru-cache==1.5,backports.ssl-match-hostname==3.5.0.1,blivet==0.61.15.69,chardet==2.2.1,configparser==3.5.0,coverage==4.4.1,decorator==3.4.0,enum34==1.1.6,funcsigs==1.0.2,futures==3.2.0,iniparse==0.4,ioprocess==1.1.0,ipaddress==1.0.16,IPy==0.75,isort==4.3.4,kitchen==1.1.1,lazy-object-proxy==1.3.1,libvirt-python==3.9.0,Magic-file-extensions==0.2,mccabe==0.6.1,mock==2.0.0,netaddr==0.7.18,ovirt-imageio-common==1.4.1,pbr==3.1.1,pluggy==0.6.0,policycoreutils-default-encoding==0.1,pthreading==0.1.3,py==1.5.4,pycurl==7.19.0,pygpgme==0.3,pyinotify==0.9.4,pykickstart==1.99.66.18,pyliblzma==0.5.3,pylint==1.8.0,pyparted==3.9,python-augeas==0.5.0,python-dateutil==2.4.2,pyudev==0.15,pyxattr==0.5.1,PyYAML==3.10,requests==2.6.0,sanlock-python==3.6.0,seobject==0.1,sepolicy==1.1,singledispatch==3.4.0.3,six==1.11.0,subprocess32==3.2.6,tox==2.9.1,urlgrabber==3.10,urllib3==1.10.2,virtualenv==16.0.0,WebOb==1.2.3,wrapt==1.10.11,yum-metadata-parser==1.1.4*14:19:22* pylint runtests: PYTHONHASHSEED='528910716'*14:19:22* pylint runtests: commands[0] | pylint --errors-only static/usr/share/vdsm/sitecustomize.py lib/vdsm lib/vdsmclient lib/yajsonrpc*14:19:23* Problem importing module base.pyc: cannot import name get_node_last_lineno*14:19:24* Problem importing module base.py: cannot import name get_node_last_lineno*14:19:34* 06-27 14:18:09,674 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/7 (executor:290)*14:19:34* 2018-06-27 14:18:09,675 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/5 (executor:290)*14:19:34* 2018-06-27 14:18:09,675 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/2 (executor:290)*14:19:34* 2018-06-27 14:18:09,676 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/3 (executor:290)*14:19:34* 2018-06-27 14:18:09,676 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/4 (executor:290)*14:19:34* 2018-06-27 14:18:09,677 DEBUG (MainThread) [Scheduler] Stopping scheduler test.Scheduler (schedule:110)*14:19:34* 2018-06-27 14:18:09,678 DEBUG (test.Scheduler) [Scheduler] stopped (schedule:143)*14:19:34* 2018-06-27 14:18:09,679 DEBUG (test.Scheduler) [Scheduler] FINISH thread <Thread(test.Scheduler, started daemon 140562082535168)> (concurrent:196)*14:19:34* --------------------- >> end captured logging << ---------------------

4 10

ovirt.org documentation refresh
by Greg Sheremeta 27 Sep '18

27 Sep '18

Hi all, Brian Proffitt has kindly offered to refresh the documentation on ovirt.org to version 4.2 (it's currently a mix of 4.0 + whatever people have updated since.) Other than the main 'official' documentation, I noticed that there is a lot of documentation on separate sites. For example, imageio, mom, all of the OST stuff, etc. For now there are no plans for any of this extra stuff, but in the future it should probably all look the same. Can you share ovirt documentation sites that you know of? Best wishes, Greg -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA <https://www.redhat.com/> gshereme(a)redhat.com IRC: gshereme <https://red.ht/sig>

5 4

s390x (mainframe) CI node is currently not available
by Barak Korren 30 Aug '18

30 Aug '18

Hi all, Unfortunately, the s390x build host we've been using had gone off-line a couple of days ago, and the ETA for having it back up is unknown. The implication of having the host down is that projects that build on s390x have their builds be stuck forever waiting for the s390x node to become available. Sine the s390x builds are missing, builds for other platforms, while being successful do not get passed to the change queue and as a result are not being released to the nightly repos. As a work around for the current situation, we need to disable the s390x builds. Here is a patch that does this for vdsm: https://gerrit.ovirt.org/c/93316/ For V2 projects the change should be done in the projects own repo, for example here is how we did it for the CI jobs of the 'jenkins' repo itself: https://gerrit.ovirt.org/c/93288/ -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted

2 1

Re: Image Transfer mechanism queries/API support
by Ranjit DSouza 28 Aug '18

28 Aug '18

Hello oVirt Development team Not sure you got a chance to see this email, resending it. Thanks Ranjit From: Ranjit DSouza Sent: Monday, July 9, 2018 1:16 PM To: 'Nir Soffer' <nsoffer(a)redhat.com> Cc: devel <devel(a)ovirt.org>; DL-VTAS-ENG-NBU-EverestFalcons <DL-VTAS-ENG-NBU-EverestFalcons(a)veritas.com>; Navin Tah <Navin.Tah(a)veritas.com>; Sudhakar Paulzagade <Sudhakar.Paulzagade(a)veritas.com>; Pavan Chavva; Yaniv Lavi (Dary) <ylavi(a)redhat.com>; Nisan, Tal <tnisan(a)redhat.com>; Daniel Erez <derez(a)redhat.com> Subject: RE: [EXTERNAL] Re: [ovirt-devel] Image Transfer mechanism queries/API support Hi Nir Thanks for getting back! We have few follow up questions: 1. On the new ‘Allocated extents API’: Can you share the release timeline for 4.2.z? From the link below it seems like it will be available on 7/30/2018. https://www.ovirt.org/develop/release-management/releases/4.2.z/release-man… However, we thought we would double check on this. 2. If this will be available in 4.2.z, does It mean, we can assume it will be backported to 4.3 also? 3. When we downloaded the snapshot disk using Image Transfer API, the resulting format of the disk is “raw”. However, for upload, we must upload a qcow2 disk (to enable further snapshots). It means, we need to convert it first using qemu-img convert. Or is there a way we directly ask via API for a qcow2 instead? Portion of the response of “GET /storagedomains/{storagedomain:id}/disksnapshots” "format" : "raw", "shareable" : "false", "sparse" : "true", "status" : "ok", "snapshot" : { "id" : "4756036e-92aa-4ebb-ae4b-052a30cd5109" }, "actual_size" : "1345228800", "content_type" : "data", "propagate_errors" : "false", "provisioned_size" : "21474836480", "storage_type" : "image", "total_size" : "0", "wipe_after_delete" : "false", 4. When downloading a snapshot in chunks, Is there any recommended chunk size? For our study we used 512 KB. Checked the documentation too. 5. Regarding your ask on ‘Can you file RFE for this, explaining the use case and the current performance’. Since you do not recommend direct access to NFS storage, we will consider it only if we see significant performance degradation using http download. Also, if time permits, we may check if there is a significant performance benefit using Unix socket, over http download (via Rest API). But as it stands now, getting the allocated extents support (soon), will alleviate most of our performance concerns. Thanks Ranjit From: Nir Soffer [mailto:nsoffer@redhat.com] Sent: Tuesday, July 3, 2018 4:29 PM To: Ranjit DSouza <Ranjit.DSouza(a)veritas.com<mailto:Ranjit.DSouza@veritas.com>> Cc: devel <devel(a)ovirt.org<mailto:devel@ovirt.org>>; DL-VTAS-ENG-NBU-EverestFalcons <DL-VTAS-ENG-NBU-EverestFalcons(a)veritas.com<mailto:DL-VTAS-ENG-NBU-EverestFalcons@veritas.com>>; Navin Tah <Navin.Tah(a)veritas.com<mailto:Navin.Tah@veritas.com>>; Sudhakar Paulzagade <Sudhakar.Paulzagade(a)veritas.com<mailto:Sudhakar.Paulzagade@veritas.com>>; Pavan Chavva; Yaniv Lavi (Dary) <ylavi(a)redhat.com<mailto:ylavi@redhat.com>>; Nisan, Tal <tnisan(a)redhat.com<mailto:tnisan@redhat.com>>; Daniel Erez <derez(a)redhat.com<mailto:derez@redhat.com>> Subject: [EXTERNAL] Re: [ovirt-devel] Image Transfer mechanism queries/API support On Tue, Jul 3, 2018 at 11:32 AM Ranjit DSouza <Ranjit.DSouza(a)veritas.com<mailto:Ranjit.DSouza@veritas.com>> wrote: ... We had a conversation with Pavan Chavva for supporting RHV. He had suggested to contact you with queries related to oVirt APIs we plan to use. We have following queries: 1. While downloading a snapshot disk, can we identify allocated extents and download only those using oVirt API? We are able to download the disk using the Image Transfer API mechanism. However, this method downloads the entire disk including the non-allocated extents, which is a performance overhead. If this functionality does not exist at this point will it be available in near future? Hi Ranjit, There is no way to do this in current 4.2, but we plan to introduce in in 4.2.z. The API will be something like: GET /images/xxx-yyy/map ... [{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": 0}, { "start": 65536, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": 65536}, { "start": 1048576, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": 1048576}, { "start": 1114112, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": 1114112}, ... { "start": 5465571328, "length": 22675456, "depth": 0, "zero": false, "data": true, "offset": 5465571328}, { "start": 5488246784, "length": 954138624, "depth": 0, "zero": true, "data": false, "offset": 5488246784}, { "start": 6442385408, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": 6442385408}] This is basically what you get using qemu-img map. You can test play this with this using: virt-builder Fedora-27 -o /var/tmp/fedora-27.img qemu-img map -f raw --output json /var/tmp/fedora-27.img This is the first data segment: { "start": 0, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": 0} This is a hole between the first data segment and the second: { "start": 65536, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": 65536} This is the second data segment: { "start": 1048576, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": 1048576} Based on this output, you will be able to get the allocated parts of the image using: Request: GET /image/xxx-yyy HTTP/1.1 Range: bytes=0-65535 Response: HTTP/1.1 206 Partial Content Content-Range: bytes 0-65535/6442450944 <data of first segment> Request: GET /image/xxx-yyy HTTP/1.1 Range: bytes=1048576-1114111 Response: HTTP/1.1 206 Partial Content Content-Range: bytes 1048576-1114111/6442450944 <data of second segment> And so on. If you create a sparse file on your backup media, and download and write the data segments at the correct offset, you will get the a sparse version of the image as on the server side. This will work for raw or qcow2 images on NFS >= 4.2, or for qcow2 images on block storage. For older NFS versions, or raw images on block storage, we can solve the issue by reading the entire image and detecting zeroes - which is quite expensive, so I'm not sure we will implement this, maybe it will be done later. We have experimental patch using special sparse format, that can support this use case, downloading entire image in one pass. This commit message explain the format: https://gerrit.ovirt.org/#/c/85413/12//COMMIT_MSG For more info on using random I/O APIs, see: http://ovirt.github.io/ovirt-imageio/random-io (available since 4.2.3) For example code uploading sparse images see: https://github.com/oVirt/ovirt-imageio/blob/master/examples/upload For best performance, you should run your application on a oVirt host, using unix socket to communicate with imageio. See: http://ovirt.github.io/ovirt-imageio/unix-socket (will be available in 4.2.5) All this will work only for the non-active layer in a qcow2 chain. We are working now on incremental backup which will allow the same for the active layer with a running vm. This is expected in 4.3, and we may have a tech preview at some point in 4.2.z. Incremantal backup will use the similar API, allowing detection of dirty parts of an image, so you can download only the data that was changed since the last backup. Please watch and comment on the feature page: https://ovirt.org/develop/release-management/features/storage/incremental-b… We are also considering exposing images using NBD. This will allow downloading and uploading images using qemu-img from any host. This work depends on TLS-PSK support in qemu-img and qemu-nbd. You can follow this work here: https://lists.nongnu.org/archive/html/qemu-devel/2018-06/threads.html#08491 2. Is there an alternate method to transfer a snapshot to and from RHV storage? Are there other methods such as NFS share where we can download snapshot image to and from RHV storage? We don't support direct access to storage by 3rd party. You should use imageio API. Can you file RFE for this, explaning the use case and the current performance issues you experience? Nir

4 4

Issue with engine-setup on master
by Ravi Shankar Nori 02 Aug '18

02 Aug '18

Hi, I pulled the latest mater rpms from nightly builds [1] and there seems to be a issue running engine-setup. engine-setup fails with message [root@laptop ovirt-engine-master-new9]# engine-setup ***L:ERROR Internal error: No module named 'async_tasks_map' I had the same issue with dev env on master Any ideas Thanks Ravi [1] https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm

4 8

Extended DPDK support in oVirt[0]
by Leon Goldberg 31 Jul '18

31 Jul '18

Hi, oVirt support for DPDK was introduced in 2017[1], and is now enhanced in terms of deployment via Ansible[2] and usage via OVN. Compatible hosts and NICs can now be easily configured to use userspace drivers, while usage is enhanced via OVN, allowing for both increased performance and SDN usability. For additional information and instructions, please refer to this blog post[0]. [0] https://www.ovirt.org/blog/2018/07/ovn-dpdk/ [1] https://www.ovirt.org/blog/2017/09/ovs-dpdk/ [2] https://github.com/ovirt/ovirt-ansible-dpdk-setup/

1 0

Lago oVirt plugin v0.45 Release Announcement
by Gal Ben Haim 31 Jul '18

31 Jul '18

Hi All, On behalf of the Lago team, I'm pleased to announce Lago oVirt plugin v0.45 is available! For a detailed release announcement, please refer to our Github page at: https://github.com/lago-project/lago-ost-plugin/releases/tag/0.45 Resources ========== Docs: http://lago-ost-plugin.readthedocs.io/en/latest/ GitHub: https://github.com/lago-project/lago-ost-plugin YUM Repository: http://resources.ovirt.org/repos/lago/stable/0.0/rpm/ OST Docs: http://ovirt-system-tests.readthedocs.io/en/latest/ Changelog: http://lago-ost-plugin.readthedocs.io/en/latest/_static/ChangeLog.txt As always, if you find any issues, or willing to contribute, visit our GitHub page. Enjoy! -- *GAL bEN HAIM* RHV DEVOPS

1 0

oVirt Dashboard issues
by Hetz Ben Hamo 31 Jul '18

31 Jul '18

Hi, Whats up with the oVirt Dashboard? it has few problems:. You can see my Dashboard here: https://imgur.com/a/n4IWppe * the errors/warning cube doesn't update so much even after cleaning and waiting an hour or 2 * Cluster cube shows "1 cluster" but also "N/A" even when my cluster is up with 2 machines (with DNS names). * The memory calculation is wrong - it still shows 503.5 GB even if 1 node is down for maintenance (which means I need to see something like 220GB) Should I open bugs on all of them in 1 bugzilla? if so, on which product? Thanks, Hetz

8 13

Image Transfer mechanism queries/API support
by Ranjit DSouza 31 Jul '18

31 Jul '18

Hello oVirt Development team We had a conversation with Pavan Chavva for supporting RHV. He had suggested to contact you with queries related to oVirt APIs we plan to use. We have following queries: 1. While downloading a snapshot disk, can we identify allocated extents and download only those using oVirt API? We are able to download the disk using the Image Transfer API mechanism. However, this method downloads the entire disk including the non-allocated extents, which is a performance overhead. If this functionality does not exist at this point will it be available in near future? 2. Is there an alternate method to transfer a snapshot to and from RHV storage? Are there other methods such as NFS share where we can download snapshot image to and from RHV storage? Thanks Ranjit Team EverestFalcons

10 13

Re: [ovirt-users] VM User with UserRole missing permissions to activate console and other actions
by Greg Sheremeta 30 Jul '18

30 Jul '18

Adding some people who may be able to help. On Wed, Jul 18, 2018 at 7:15 AM Callum Smith <callum(a)well.ox.ac.uk> wrote: > Dear All, > > Please see the errors below. I'm seeing this in the engine.log when as a > user I'm trying to activate either a VM console or reboot a VM which I have > access to as a user ("UserRole permission assigned to VM). > > 2018-07-18 10:51:33,554+01 INFO > [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default task-9) > [557ca876] Running command > : CreateUserSessionCommand internal: false. > 2018-07-18 10:51:33,575+01 INFO > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-9) [557ca876] E > VENT_ID: USER_VDC_LOGIN(30), User callum@Biomedical Research Computing > connecting from '192.168.1.241' using session 'wiWA25wdaRP1zay > iyTSGBJKpvi89LdzgKqeX12BcZhNVhpV2BIA+zkAnT50xOSDglxnhfAi3S2ZiODls8JYFUA==' > logged in. > 2018-07-18 10:51:34,135+01 ERROR > [org.ovirt.engine.core.bll.GetSystemStatisticsQuery] (default task-5) > [8d830cdb-fc11-4e68-94e6-73309 > 65c4488] Query execution failed due to insufficient permissions. > 2018-07-18 10:51:34,205+01 ERROR > [org.ovirt.engine.core.bll.GetPermissionsForObjectQuery] (default task-26) > [ba1825f1-60fb-44cd-8b57- > ea701cf698c0] Query execution failed due to insufficient permissions. > 2018-07-18 10:51:34,242+01 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default > task-26) [] Operation Faile > d: query execution failed due to insufficient permissions. > 2018-07-18 10:51:34,389+01 ERROR > [org.ovirt.engine.core.bll.storage.domain.GetStorageDomainListByIdQuery] > (default task-17) [02965366 > -44b0-4370-ab83-4781065e46c2] Query execution failed due to insufficient > permissions. > 2018-07-18 10:51:34,393+01 ERROR > [org.ovirt.engine.core.bll.storage.domain.GetStorageDomainListByIdQuery] > (default task-17) [02965366 > -44b0-4370-ab83-4781065e46c2] Query execution failed due to insufficient > permissions. > 2018-07-18 10:51:34,394+01 ERROR > [org.ovirt.engine.core.bll.storage.domain.GetStorageDomainListByIdQuery] > (default task-17) [02965366 > -44b0-4370-ab83-4781065e46c2] Query execution failed due to insufficient > permissions. > 2018-07-18 10:51:34,396+01 ERROR > [org.ovirt.engine.core.bll.storage.domain.GetStorageDomainListByIdQuery] > (default task-17) [02965366 > -44b0-4370-ab83-4781065e46c2] Query execution failed due to insufficient > permissions. > 2018-07-18 10:51:59,195+01 WARN > [org.ovirt.engine.core.bll.SetVmTicketCommand] (default task-18) > [7881a832] User '9386d6f5-f172-4cdb > -abca-62492a357888' is trying to take the console of virtual machine > 'ddb23e0a-01d5-403c-89ab-37c400d2c938', but the console is alrea > dy taken by user 'd021fc10-4f7c-11e8-88cb-00163e6a7aff'. > 2018-07-18 10:51:59,197+01 INFO > [org.ovirt.engine.core.bll.SetVmTicketCommand] (default task-18) > [7881a832] No permission found for > user '9386d6f5-f172-4cdb-abca-62492a357888' or one of the groups he is > member of, when running action 'SetVmTicket', Required permiss > ions are: Action type: 'USER' Action group: 'RECONNECT_TO_VM' Object type: > 'VM' Object ID: 'ddb23e0a-01d5-403c-89ab-37c400d2c938'. > 2018-07-18 10:51:59,197+01 WARN > [org.ovirt.engine.core.bll.SetVmTicketCommand] (default task-18) > [7881a832] Validation of action 'Se > tVmTicket' failed for user callum@Biomedical Research Computing. Reasons: > VAR__ACTION__SET,VAR__TYPE__VM_TICKET,USER_CANNOT_FORCE_REC > ONNECT_TO_VM > 2018-07-18 10:51:59,198+01 ERROR > [org.ovirt.engine.api.restapi.resource.BackendVmGraphicsConsoleResource] > (default task-18) [] Operat > ion Failed: USER_CANNOT_FORCE_RECONNECT_TO_VM > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. callum(a)well.ox.ac.uk > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/D7GSJDZ32DBIM… > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA <https://www.redhat.com/> gshereme(a)redhat.com IRC: gshereme <https://red.ht/sig>

4 5