September 2018 - Devel - oVirt List Archives

[VDSM] Started getting virNetTLSContextCheckCertTimes:154 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem has expired
by Edward Haas 11 Jan '24

11 Jan '24

Hi, I have a VM which acts as an oVirt host with VDSM installed. I use it for testing without connecting it to Engine. Recently VDSM fails to come up and I see this error: Sep 26 17:58:18 localhost libvirtd: 2018-09-26 14:58:18.978+0000: 12176: error : virNetTLSContextCheckCertTimes:154 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem has expired Any hint on how to generate or fetch a new certificate without the help of Engine? Thanks, Edy.

3 5

Error Java SDK Issue??
by Geschwentner, Patrick 02 Apr '21

02 Apr '21

Dear Ladies and Gentlemen! I am currently working with the java-sdk and I encountered a problem. If I would like to retrieve the disk details, I get the following error: Disk currDisk = ovirtConnection.followLink(diskAttachment.disk()); The Error is occurring in this line: [cid:image001.png@01D44537.AF127FD0] The getResponst looks quiet ok. (I inspected: [cid:image002.png@01D44537.AF127FD0] and it looks ok). Error: wrong number of arguments The code is quiet similar to what you published on github (https://github.com/oVirt/ovirt-engine-sdk-java/blob/master/sdk/src/test/jav… ). Can you confirm the defect? Best regards Patrick

6 11

[VDSM] test_echo(1024, False) (stomp_test.StompTests) fails
by Nir Soffer 01 Dec '18

01 Dec '18

This tests used to fail in the past, but since we fixed it or the code it never failed. Maybe the slave was overloaded? *14:19:02* ======================================================================*14:19:02* ERROR: *14:19:02* ----------------------------------------------------------------------*14:19:02* Traceback (most recent call last):*14:19:02* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/testlib.py", line 143, in wrapper*14:19:02* return f(self, *args)*14:19:02* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/stomp_test.py", line 95, in test_echo*14:19:02* str(uuid4())),*14:19:02* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/lib/yajsonrpc/jsonrpcclient.py", line 77, in callMethod*14:19:02* raise exception.JsonRpcNoResponseError(method=methodName)*14:19:02* JsonRpcNoResponseError: No response for JSON-RPC request: {'method': 'echo'}*14:19:02* -------------------- >> begin captured logging << --------------------*14:19:02* 2018-06-27 14:17:54,524 DEBUG (MainThread) [vds.MultiProtocolAcceptor] Creating socket (host='::1', port=0, family=10, socketype=1, proto=6) (protocoldetector:225)*14:19:02* 2018-06-27 14:17:54,526 INFO (MainThread) [vds.MultiProtocolAcceptor] Listening at ::1:36713 (protocoldetector:183)*14:19:02* 2018-06-27 14:17:54,535 DEBUG (MainThread) [Scheduler] Starting scheduler test.Scheduler (schedule:98)*14:19:02* 2018-06-27 14:17:54,537 DEBUG (test.Scheduler) [Scheduler] START thread <Thread(test.Scheduler, started daemon 140562082535168)> (func=<bound method Scheduler._run of <vdsm.schedule.Scheduler object at 0x7fd74ca00390>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,538 DEBUG (test.Scheduler) [Scheduler] started (schedule:140)*14:19:02* 2018-06-27 14:17:54,546 DEBUG (JsonRpc (StompReactor)) [root] START thread <Thread(JsonRpc (StompReactor), started daemon 140562629256960)> (func=<bound method StompReactor.process_requests of <yajsonrpc.stompserver.StompReactor object at 0x7fd74ca006d0>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,547 DEBUG (MainThread) [Executor] Starting executor (executor:128)*14:19:02* 2018-06-27 14:17:54,549 DEBUG (MainThread) [Executor] Starting worker jsonrpc/0 (executor:286)*14:19:02* 2018-06-27 14:17:54,553 DEBUG (jsonrpc/0) [Executor] START thread <Thread(jsonrpc/0, started daemon 140562612471552)> (func=<bound method _Worker._run of <Worker name=jsonrpc/0 waiting task#=0 at 0x7fd74ca00650>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,554 DEBUG (jsonrpc/0) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,557 DEBUG (MainThread) [Executor] Starting worker jsonrpc/1 (executor:286)*14:19:02* 2018-06-27 14:17:54,558 DEBUG (MainThread) [Executor] Starting worker jsonrpc/2 (executor:286)*14:19:02* 2018-06-27 14:17:54,559 DEBUG (jsonrpc/2) [Executor] START thread <Thread(jsonrpc/2, started daemon 140562620864256)> (func=<bound method _Worker._run of <Worker name=jsonrpc/2 waiting task#=0 at 0x7fd74c9fb350>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,560 DEBUG (jsonrpc/2) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,561 DEBUG (MainThread) [Executor] Starting worker jsonrpc/3 (executor:286)*14:19:02* 2018-06-27 14:17:54,562 DEBUG (jsonrpc/3) [Executor] START thread <Thread(jsonrpc/3, started daemon 140562124498688)> (func=<bound method _Worker._run of <Worker name=jsonrpc/3 waiting task#=0 at 0x7fd74bc80290>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,563 DEBUG (jsonrpc/3) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,564 DEBUG (MainThread) [Executor] Starting worker jsonrpc/4 (executor:286)*14:19:02* 2018-06-27 14:17:54,565 DEBUG (jsonrpc/4) [Executor] START thread <Thread(jsonrpc/4, started daemon 140562116105984)> (func=<bound method _Worker._run of <Worker name=jsonrpc/4 waiting task#=0 at 0x7fd74bc80790>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,566 DEBUG (jsonrpc/4) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,568 DEBUG (jsonrpc/1) [Executor] START thread <Thread(jsonrpc/1, started daemon 140562132891392)> (func=<bound method _Worker._run of <Worker name=jsonrpc/1 waiting task#=0 at 0x7fd74c9fb690>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,569 DEBUG (jsonrpc/1) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,569 DEBUG (MainThread) [Executor] Starting worker jsonrpc/5 (executor:286)*14:19:02* 2018-06-27 14:17:54,570 DEBUG (jsonrpc/5) [Executor] START thread <Thread(jsonrpc/5, started daemon 140562107713280)> (func=<bound method _Worker._run of <Worker name=jsonrpc/5 waiting task#=0 at 0x7fd74bc80090>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,571 DEBUG (jsonrpc/5) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,572 DEBUG (MainThread) [Executor] Starting worker jsonrpc/6 (executor:286)*14:19:02* 2018-06-27 14:17:54,580 DEBUG (MainThread) [Executor] Starting worker jsonrpc/7 (executor:286)*14:19:02* 2018-06-27 14:17:54,603 DEBUG (jsonrpc/6) [Executor] START thread <Thread(jsonrpc/6, started daemon 140562099320576)> (func=<bound method _Worker._run of <Worker name=jsonrpc/6 waiting task#=0 at 0x7fd74d0dacd0>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,604 DEBUG (jsonrpc/6) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,615 DEBUG (jsonrpc/7) [Executor] START thread <Thread(jsonrpc/7, started daemon 140562090927872)> (func=<bound method _Worker._run of <Worker name=jsonrpc/7 waiting task#=0 at 0x7fd74bfe0310>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,616 DEBUG (JsonRpcServer) [root] START thread <Thread(JsonRpcServer, started daemon 140561730238208)> (func=<bound method JsonRpcServer.serve_requests of <yajsonrpc.JsonRpcServer object at 0x7fd74ca00690>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,618 DEBUG (MainThread) [vds.MultiProtocolAcceptor] Adding detector <yajsonrpc.stompserver.StompDetector instance at 0x7fd74c28cc68> (protocoldetector:210)*14:19:02* 2018-06-27 14:17:54,625 INFO (Detector thread) [ProtocolDetector.AcceptorImpl] Accepted connection from ::1:52432 (protocoldetector:61)*14:19:02* 2018-06-27 14:17:54,626 DEBUG (Detector thread) [ProtocolDetector.Detector] Using required_size=11 (protocoldetector:89)*14:19:02* 2018-06-27 14:17:54,627 DEBUG (jsonrpc/7) [Executor] Worker started (executor:298)*14:19:02* 2018-06-27 14:17:54,634 DEBUG (MainThread) [jsonrpc.AsyncoreClient] Sending response (stompclient:293)*14:19:02* 2018-06-27 14:17:54,634 DEBUG (Client ::1:36713) [root] START thread <Thread(Client ::1:36713, started daemon 140561713452800)> (func=<bound method Reactor.process_requests of <yajsonrpc.betterAsyncore.Reactor object at 0x7fd74bdb1d90>>, args=(), kwargs={}) (concurrent:193)*14:19:02* 2018-06-27 14:17:54,641 INFO (Detector thread) [ProtocolDetector.Detector] Detected protocol stomp from ::1:52432 (protocoldetector:125)*14:19:02* 2018-06-27 14:17:54,645 INFO (Detector thread) [Broker.StompAdapter] Processing CONNECT request (stompserver:94)*14:19:02* 2018-06-27 14:17:54,648 DEBUG (Client ::1:36713) [yajsonrpc.protocols.stomp.AsyncClient] Stomp connection established (stompclient:137)*14:19:02* 2018-06-27 14:17:54,651 DEBUG (jsonrpc/0) [jsonrpc.JsonRpcServer] Calling 'echo' in bridge with [u'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut en'] (__init__:328)*14:19:02* 2018-06-27 14:17:54,652 DEBUG (jsonrpc/0) [jsonrpc.JsonRpcServer] Return 'echo' in bridge with Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut en (__init__:355)*14:19:02* 2018-06-27 14:17:54,653 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call echo succeeded in 0.01 seconds (__init__:311)*14:19:02* 2018-06-27 14:17:54,661 INFO (Detector thread) [Broker.StompAdapter] Subscribe command received (stompserver:123)*14:19:02* 2018-06-27 14:17:54,662 DEBUG (Detector thread) [protocoldetector.StompDetector] Stomp detected from ('::1', 52432) (stompserver:412)*14:19:02* 2018-06-27 14:18:09,649 DEBUG (MainThread) [vds.MultiProtocolAcceptor] Stopping Acceptor (protocoldetector:214)*14:19:02* 2018-06-27 14:18:09,650 INFO (MainThread) [jsonrpc.JsonRpcServer] Stopping JsonRPC Server (__init__:441)*14:19:02* 2018-06-27 14:18:09,651 DEBUG (MainThread) [Executor] Stopping executor (executor:137)*14:19:02* 2018-06-27 14:18:09,652 DEBUG (Client ::1:36713) [root] FINISH thread <Thread(Client ::1:36713, started daemon 140561713452800)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,652 DEBUG (JsonRpcServer) [root] FINISH thread <Thread(JsonRpcServer, started daemon 140561730238208)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,654 DEBUG (JsonRpc (StompReactor)) [root] FINISH thread <Thread(JsonRpc (StompReactor), started daemon 140562629256960)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,655 DEBUG (jsonrpc/2) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,658 DEBUG (jsonrpc/3) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,659 DEBUG (jsonrpc/4) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,660 DEBUG (jsonrpc/1) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,660 DEBUG (jsonrpc/5) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,660 DEBUG (jsonrpc/6) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,665 DEBUG (jsonrpc/6) [Executor] FINISH thread <Thread(jsonrpc/6, started daemon 140562099320576)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,661 DEBUG (jsonrpc/7) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,662 DEBUG (jsonrpc/2) [Executor] FINISH thread <Thread(jsonrpc/2, started daemon 140562620864256)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,663 DEBUG (jsonrpc/3) [Executor] FINISH thread <Thread(jsonrpc/3, started daemon 140562124498688)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,663 DEBUG (jsonrpc/1) [Executor] FINISH thread <Thread(jsonrpc/1, started daemon 140562132891392)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,664 DEBUG (jsonrpc/4) [Executor] FINISH thread <Thread(jsonrpc/4, started daemon 140562116105984)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,664 DEBUG (jsonrpc/0) [Executor] Worker stopped (executor:303)*14:19:02* 2018-06-27 14:18:09,665 DEBUG (jsonrpc/5) [Executor] FINISH thread <Thread(jsonrpc/5, started daemon 140562107713280)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,667 DEBUG (jsonrpc/7) [Executor] FINISH thread <Thread(jsonrpc/7, started daemon 140562090927872)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,667 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/0 (executor:290)*14:19:02* 2018-06-27 14:18:09,671 DEBUG (jsonrpc/0) [Executor] FINISH thread <Thread(jsonrpc/0, started daemon 140562612471552)> (concurrent:196)*14:19:02* 2018-06-27 14:18:09,673 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/1 (executor:290)*14:19:02* 2018-06-27 14:18:09,674 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/6 (executor:290)*14:19:02* 2018-Coverage.py warning: Module /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/vdsm was never imported. (module-not-imported)*14:19:05* pylint installdeps: pylint==1.8*14:19:22* pylint installed: The directory '/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.,astroid==1.6.5,backports.functools-lru-cache==1.5,backports.ssl-match-hostname==3.5.0.1,blivet==0.61.15.69,chardet==2.2.1,configparser==3.5.0,coverage==4.4.1,decorator==3.4.0,enum34==1.1.6,funcsigs==1.0.2,futures==3.2.0,iniparse==0.4,ioprocess==1.1.0,ipaddress==1.0.16,IPy==0.75,isort==4.3.4,kitchen==1.1.1,lazy-object-proxy==1.3.1,libvirt-python==3.9.0,Magic-file-extensions==0.2,mccabe==0.6.1,mock==2.0.0,netaddr==0.7.18,ovirt-imageio-common==1.4.1,pbr==3.1.1,pluggy==0.6.0,policycoreutils-default-encoding==0.1,pthreading==0.1.3,py==1.5.4,pycurl==7.19.0,pygpgme==0.3,pyinotify==0.9.4,pykickstart==1.99.66.18,pyliblzma==0.5.3,pylint==1.8.0,pyparted==3.9,python-augeas==0.5.0,python-dateutil==2.4.2,pyudev==0.15,pyxattr==0.5.1,PyYAML==3.10,requests==2.6.0,sanlock-python==3.6.0,seobject==0.1,sepolicy==1.1,singledispatch==3.4.0.3,six==1.11.0,subprocess32==3.2.6,tox==2.9.1,urlgrabber==3.10,urllib3==1.10.2,virtualenv==16.0.0,WebOb==1.2.3,wrapt==1.10.11,yum-metadata-parser==1.1.4*14:19:22* pylint runtests: PYTHONHASHSEED='528910716'*14:19:22* pylint runtests: commands[0] | pylint --errors-only static/usr/share/vdsm/sitecustomize.py lib/vdsm lib/vdsmclient lib/yajsonrpc*14:19:23* Problem importing module base.pyc: cannot import name get_node_last_lineno*14:19:24* Problem importing module base.py: cannot import name get_node_last_lineno*14:19:34* 06-27 14:18:09,674 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/7 (executor:290)*14:19:34* 2018-06-27 14:18:09,675 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/5 (executor:290)*14:19:34* 2018-06-27 14:18:09,675 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/2 (executor:290)*14:19:34* 2018-06-27 14:18:09,676 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/3 (executor:290)*14:19:34* 2018-06-27 14:18:09,676 DEBUG (MainThread) [Executor] Waiting for worker jsonrpc/4 (executor:290)*14:19:34* 2018-06-27 14:18:09,677 DEBUG (MainThread) [Scheduler] Stopping scheduler test.Scheduler (schedule:110)*14:19:34* 2018-06-27 14:18:09,678 DEBUG (test.Scheduler) [Scheduler] stopped (schedule:143)*14:19:34* 2018-06-27 14:18:09,679 DEBUG (test.Scheduler) [Scheduler] FINISH thread <Thread(test.Scheduler, started daemon 140562082535168)> (concurrent:196)*14:19:34* --------------------- >> end captured logging << ---------------------

4 10

Request for reviews - vdsm-tool update-volume
by Germano Veit Michel 29 Nov '18

29 Nov '18

Hi, I'm working an a tool (vdsm-tool update-volume) to make modifying SD metadata easier and more importantly, safer. This is very useful to recover from failed LSMs or snapshot issues. The plan is to use the VDSM API (modified by some of these patches) and add a tool (vdsm-tool) that talks to the API and modifies the volumes metadata as required by the user. Currently this is done manually, i.e.: looking at MD_XXX tags, doing dd, sed and then dd back to the storage. Any wrong argument (like a skip in place of a seek) can ruin the entire metadata, so this tool can be quite handy. The code is not necessarily 100% finished yet, but I've been testing this for some time and it seems ok from a functional point of view. I'm just not sure everything I did (especially inside VDSM, example 94366) is correct. Your comments on what can/should be improved are very welcome at this point. Please see this series and help reviewing it. https://gerrit.ovirt.org/#/q/topic:update-volume+(status:open+OR+status:mer… https://gerrit.ovirt.org/#/c/93258/ Thanks, Germano

2 9

[ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 14-09-2018 ] [ 002_bootstrap.add_hosts ]
by Dafna Ron 22 Oct '18

22 Oct '18

Hi, The previous regression was resolved and we now have a new regression. I don't think that the reported change is related so can someone from ovirt-engine take a look? The failure is add host on the upgrade suite. Please note that we have not had an engine-ovirt build for over 10 days due to several consecutive regressions and I would ask you to stop merging until we can stabilize the project and have a new package of engine. error: 2018-09-14 05:51:07,670-04 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] SSH execute 'root@lago-upgrade-from-release-suite-master-host-0' 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar -b1 --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:True' 2018-09-14 05:51:08,550-04 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [5c91fcbd] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-suite-master-host-0. Stage: Initializing. 2018-09-14 05:51:08,565-04 INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (VdsDeploy) [5c91fcbd] transaction rolled back 2018-09-14 05:51:08,574-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [5c91fcbd] Error during deploy dialog 2018-09-14 05:51:08,578-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Error during host lago-upgrade-from-release-suite-master-host-0 install 2018-09-14 05:51:08,586-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during installation of Host lago-upgrade-from-release-suite-master-host-0: CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested exception is org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00. 2018-09-14 05:51:08,586-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Error during host lago-upgrade-from-release-suite-master-host-0 install, preferring first exception: CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested exception is org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00 2018-09-14 05:51:08,586-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Host installation failed for host 'e475e93a-63b3-4573-b242-162c2ed864f0', 'lago-upgrade-from-release-suite-master-host-0': CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested exception is org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00 2018-09-14 05:51:08,615-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] START, SetVdsStatusVDSCommand(HostName = lago-upgrade-from-release-suite-master-host-0, SetVdsStatusVDSCommandParameters:{hostId='e475e93a-63b3-4573-b242-162c2ed864f0', status='InstallFailed', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 146cdc08 2018-09-14 05:51:08,626-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] FINISH, SetVdsStatusVDSCommand, return: , log id: 146cdc08 2018-09-14 05:51:08,639-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] EVENT_ID: VDS_INSTALL_FAILED(505), Host lago-upgrade-from-release-suite-master-host-0 installation failed. CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested exception is org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00. 2018-09-14 05:51:08,652-04 INFO [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Lock freed to object 'EngineLock:{exclusiveLocks='[e475e93a-63b3-4573-b242-162c2ed864f0=VDS]', sharedLocks=''}' 2018-09-14 05:51:37,996-04 INFO [org.ovirt.engine.core.bll.quota.QuotaManager] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [] Quota Cache updated. (19 msec) (END) Thanks, Dafna

9 30

increasingly common OST failure: Cannot setup Networks. Another Setup Networks or Host Refresh process in progress on the host. Please try later.
by Dan Kenigsberg 02 Oct '18

02 Oct '18

FYI, Ales is working on retrying setupNetwork in case there is an async getCapabilities going on the host. https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-4.2_change-q… However, the real bug is partially hidden by Lago complaining about ExtractPathNoPathError: scp: /root/fluentd: No such file or directory What is it? Is it already tracked by someone?

4 4

Help understanding vdsHeartbeatInSeconds relation to Heartbeat exceeded
by Germano Veit Michel 02 Oct '18

02 Oct '18

Hi, I've been trying to understand why we see some heartbeat exceeded messages with a last response a very short time ago, while the vdsHeartbeatInSeconds option would allow for much more before triggering the disconnect. The vdc_option vdsHeartbeatInSeconds is set to 30 by default. It is used here in the engine, the propagates to vdsm-jsonrpc-java int heartbeat = Config.<Integer> getValue(ConfigValues.vdsHeartbeatInSeconds) * 1000; TransportFactory.createVdsServer() -> JsonRpcUtils.createStompClient() -> StompClientPolicy ClientPolicy -> createClient() -> getJsonClient() -> setClientPolicy() -> setRetryPolicy() So I understand ClientPolicy.incomingHearbeat = vdsHeartbeatInSeconds*1000 = 30000ms If this is the case, why do we see messages such as this, if isIncomingHeartbeatExceeded() shouldn't return True in case the last heartbeat was less than 30000ms ago? 2018-09-17 10:46:49,891+03 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Heartbeat exceeded for host '192.168.7.235', last response arrived 1702 ms ago. if (!this.isInInit() && this.policy.isIncomingHeartbeat() && this.isIncomingHeartbeatExceeded()) { String msg = String.format("Heartbeat exceeded for host '%s', last response arrived %s ms ago.", getHostname(), getHeartbeatTime()); log.error(msg); this.disconnect(msg); } private long getHeartbeatTime() { return System.currentTimeMillis() - this.lastIncomingHeartbeat; } private boolean isIncomingHeartbeatExceeded() { return this.lastIncomingHeartbeat + this.policy.getIncomingHeartbeat() < System.currentTimeMillis(); } public ClientPolicy(int retryTimeOut, int retryNumber, int incomingHeartbeat, int outgoingHeartbeat, List<Class<? extends Exception>> retryableExceptions) { this.retryNumber = retryNumber; this.retryTimeOut = retryTimeOut; setIncomingHeartbeat(incomingHeartbeat); setOutgoingHeartbeat(outgoingHeartbeat); this.exceptions = Collections.unmodifiableList(retryableExceptions); } I used byteman to print the value stored in policy.incomingHeartbeat and it is: If vdsHeartbeatInSeconds = 45 2018-09-27 15:17:13,673+10 INFO [stdout] (SSL Stomp Reactor) policy.incomingHeartbeat: 33750 If vdsHeartbeatInSeconds = 30 (default) 2018-09-27 15:21:51,099+10 INFO [stdout] (SSL Stomp Reactor) policy.incomingHeartbeat: 22500 I was expecting 45000 and 30000. But even with 33750 and 22500, I dont understand why we see "Heartbeat exceeded" and disconnection if the last heartbeat arrived just 1700ms ago. I'm clearly missing something. Could someone please put some light on what is the missing piece? Thanks, Germano

1 1

/root/fluentd no longer exists
by Anton Marchukov 01 Oct '18

01 Oct '18

Hello Shirly, All. We used to collect fluentd artifacts in OST from /root/fluentd, but starting from around mid night today Fri Aug 31 (or maybe late evening Thu Aug 30) the tests started to fail to locate /root/fluentd. We merged a patch to disable its collection [1] so we can let other tests run, but we need to find what happened and restore fluentd collection. I open a JIRA issue [2] on this. Do you have any idea? Thanks. [1] https://gerrit.ovirt.org/#/c/94067/ [2] https://ovirt-jira.atlassian.net/browse/OVIRT-2464 -- Anton Marchukov Team Lead - Release Management - RHV DevOps - Red Hat

4 9

new dependency for vdsm 4.2
by Marcin Sobczyk 01 Oct '18

01 Oct '18

Hi, "python-enum34" has been added as a new dependency for vdsm-4.2 after merging https://gerrit.ovirt.org/#/q/topic:vdsm-api-schema-pickle+(status:open+OR+s… . Regards, Marcin

2 2

Status update on "Hyperconverged Gluster oVirt support"
by Gobinda Das 30 Sep '18

30 Sep '18

Hi All, Status update on "Hyperconverged Gluster oVirt support" Features Completed: ================ cockpit-ovirt ------------- 1- Asymmetric brick configuration.Brick can be configured per host basis i.e. If the user wanted to make use of sdb from host1, sdc from host2, and sdd from host3. 2- Dedupe and Compression integration via VDO support (see https://github.com/dm-vdo/kvdo) Gluster bricks are created on vdo devices 3- LVM cache configuration support (Configure cache by using fast block device such as SSD drive to imrove the performance of a larger and slower logical volumes) 4- Auto addition of 2nd and 3rd hosts in a 3 node setup during deployment 5- Auto creation of storage domains based on gluster volumes created during setup 6- Single node deployment support via Cockpit UI. For details on single node deployment - https://www.ovirt.org/documentation/gluster-hyperconverged/chap-Single_node… 7- Gluster Management Dashboard (Dashboard will show the nodes in cluster,Volumes and bricks. User can expand the cluster and also can create new volume in existing cluster nodes ) oVirt ------- 1- Reset brick support from UI to allow users to replace a faulty brick 2- Create brick from engine now supports configuring an SSD device as lvmcache device when bricks are created on spinning disks 3- VDO monitoring GlusterFS --------------- Enhancements to performance with fuse by 15x 1. Cluster after eager lock change for better detection of multiple clients 2. Changing qemu option aio to "native" instead of "threads". end-to-end deployment: -------------------------------- 1- End to end deployment of a Gluster + Ovirt hyperconverged environment using ansible roles ( https://github.com/gluster/gluster-ansible/tree/master/playbooks ). The only pre-requisite is a CentOS node/oVirt node Future Plan: ========== cockpit-ovirt: 1- ansible-roles integration for deployment 2- Support for different volume types vdsm: 1- Python3 compatibility of vdsm-gluster 2- Native 4K support -- Thanks, Gobinda

4 5