------=_Part_107438_1091840694.1502716209553 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 aGVsbG8gZXZlcnlvbmUhCgoKICAgICAgIE5vdyBteSBtYWluIHdheSB0byBsZWFybiBhbmQgZGVi dWcgdmRzbSBpcyB0byB3cml0ZSBwZGIgY29kZSB0byB2ZHNtIGFuZCByZXN0YXJ0IHZkc21kLnNl cnZpY2UsIGRvc2UgaXQgaGF2ZSBhIGJldHRlciB3YXk/IEFzIHNvbWUgaWRlIHRoYXQgaXMgbW9y ZSBjb252ZW5pZW50IHRvIGFkZCBicmVha2luZyBwb2ludOOAgXN0ZXAgaW50bywgZXRjLiAKCgoK CkhhdmUgYSBuaWNlIGRheQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKCnBlbmNj CgoKCgo= ------=_Part_107438_1091840694.1502716209553 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: base64 PGh0bWw+CjxoZWFkPgogICAgPG1ldGEgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIiBjb250ZW50 PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9VVRGLTgiPgo8L2hlYWQ+Cjxib2R5Pgo8c3R5bGU+CiAgICBm b250ewogICAgICAgIGxpbmUtaGVpZ2h0OiAxLjc7CiAgICB9Cjwvc3R5bGU+CjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OiZxdW90O+W+rui9r+mbhem7kSZxdW90OzsgZm9udC1zaXplOiAxNHB4OyBj b2xvcjojMDAwMDAwOyBsaW5lLWhlaWdodDoxLjc7Ij4KICAgIAo8ZGl2PjxzcGFuIHN0eWxlPSJm b250LWZhbWlseTogVmVyZGFuYSwg5b6u6L2v6ZuF6buRLCDlrovkvZMsIHNhbnMtc2VyaWY7IGxp bmUtaGVpZ2h0OiAyMy44cHg7Ij5oZWxsbyBldmVyeW9uZSE8L3NwYW4+PGJyIHN0eWxlPSJmb250 LWZhbWlseTogVmVyZGFuYSwg5b6u6L2v6ZuF6buRLCDlrovkvZMsIHNhbnMtc2VyaWY7IGxpbmUt aGVpZ2h0OiAyMy44cHg7Ij48ZGl2IGlkPSJpc0ZvcndhcmRDb250ZW50IiBzdHlsZT0iZm9udC1m YW1pbHk6IFZlcmRhbmEsIOW+rui9r+mbhem7kSwg5a6L5L2TLCBzYW5zLXNlcmlmOyBsaW5lLWhl aWdodDogMjMuOHB4OyI+PHA+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7Tm93IG15IG1haW4g d2F5IHRvIGxlYXJuIGFuZCBkZWJ1ZyB2ZHNtIGlzIHRvIHdyaXRlIHBkYiBjb2RlIHRvIHZkc20g YW5kIHJlc3RhcnQgdmRzbWQuc2VydmljZSwgZG9zZSBpdCBoYXZlIGEgYmV0dGVyIHdheT8mbmJz cDs8c3BhbiBzdHlsZT0ibGluZS1oZWlnaHQ6IDIzLjhweDsiPkFzIHNvbWUgaWRlIHRoYXQgaXMg bW9yZSZuYnNwOzwvc3Bhbj48c3BhbiBzdHlsZT0ibGluZS1oZWlnaHQ6IDIzLjhweDsiPmNvbnZl bmllbnQgdG88L3NwYW4+PHNwYW4gc3R5bGU9ImxpbmUtaGVpZ2h0OiAyMy44cHg7Ij4mbmJzcDth ZGQ8L3NwYW4+PHNwYW4gc3R5bGU9ImxpbmUtaGVpZ2h0OiAyMy44cHg7Ij4mbmJzcDs8L3NwYW4+ PHNwYW4gY2xhc3M9ImRpY3RUYWciIHN0eWxlPSJsaW5lLWhlaWdodDogMjMuOHB4OyI+PC9zcGFu PjxzcGFuIHN0eWxlPSJsaW5lLWhlaWdodDogMjMuOHB4OyI+YnJlYWtpbmcgcG9pbnTjgIFzdGVw IGludG8sIGV0Yy4mbmJzcDs8L3NwYW4+PC9wPjxwPjxicj48L3A+PHByZSBjbGFzcz0ibW96LXNp Z25hdHVyZSIgY29scz0iNzIiIHN0eWxlPSJmb250LWZhbWlseTogVmVyZGFuYSwg5b6u6L2v6ZuF 6buRLCDlrovkvZMsIHNhbnMtc2VyaWYgIWltcG9ydGFudDsiPkhhdmUgYSBuaWNlIGRheTxicj4t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLTxicj48L3ByZT48cD48L3A+PHA+cGVuY2M8 L3A+PC9kaXY+PC9kaXY+CjxkaXY+PGJyPjwvZGl2PjxkaXYgaWQ9Im50ZXMtcGNtYWlsLXNpZ25h dHVyZSIgc3R5bGU9ImZvbnQtZmFtaWx5Oiflvq7ova/pm4Xpu5EnIj48Zm9udCBzdHlsZT0icGFk ZGluZzogMDsgbWFyZ2luOjA7Ij4gICAgICAgICAgICAgICAgPC9mb250PgoKPC9kaXY+PGJyPgo8 c3R5bGUgdHlwZT0idGV4dC9jc3MiPgogICAgICAgIGEjbnRlcy1wY21haWwtc2lnbmF0dXJlLWRl ZmF1bHQ6aG92ZXIgewogICAgICAgICAgICB0ZXh0LWRlY29yYXRpb246IHVuZGVybGluZTsKICAg ICAgICAgICAgY29sb3I6ICMzNTkzZGI7CiAgICAgICAgICAgIGN1cnNvcjogcG9pbnRlcjsKICAg ICAgICB9CiAgICA8L3N0eWxlPjwhLS3wn5iALS0+CjwvZGl2Pgo8L2JvZHk+CjwvaHRtbD4= ------=_Part_107438_1091840694.1502716209553--
On Mon, Aug 14, 2017 at 4:26 PM pengyixiang <yxpengi386@163.com> wrote:
hello everyone!
Now my main way to learn and debug vdsm is to write pdb code to vdsm and restart vdsmd.service, dose it have a better way? As some ide that is more convenient to add breaking point、step into, etc.
The best way to debug vdsm is to write good tests. When you fail to write tests or in untested areas, your best way is to use logs. Stopping vdsm in the debugger may cause engine to fence it, and in general stopping entire multi threaded program like vdsm is not very useful. Another way to debug is via manhole - you can get a python shell inside vdsm, and poke in its internals. To use manhole install it: # pip install -U manhole And enable using vdsm dropin conf - create a file: # cat /etc/vdsm/vdsm.conf.d/manhole.conf [devel] # Enable manhole debugging service (requires manhole package). manhole_enable = true And restart vdsm. To open a python shell inside vdsm you can use netcat, but socat is much better because of the readline integration. # socat readline unix-connect:/run/vdsm/vdsmd.manhole ######### ProcessID=21004, ThreadID=139830629488384 ######### File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap self.__bootstrap_inner() File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner self.run() File: "/usr/lib/python2.7/site-packages/manhole.py", line 225, in run self.handle(self.client, self.locals) File: "/usr/lib/python2.7/site-packages/manhole.py", line 268, in handle run_repl(locals) File: "/usr/lib/python2.7/site-packages/manhole.py", line 313, in run_repl dump_stacktraces() File: "/usr/lib/python2.7/site-packages/manhole.py", line 578, in dump_stacktraces for filename, lineno, name, line in traceback.extract_stack(stack): [--snipped stacktrace of all threads--] Python 2.7.5 (default, May 3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)] on linux2 Type "help", "copyright", "credits" or "license" for more information. (ManholeConsole)
Checking your local namespace:
dir() ['__builtins__', 'cif', 'dump_stacktraces', 'irs', 'os', 'socket', 'sys', 'traceback']
irs is the storage endpoint, you can look into it:
irs._obj.domainMonitor._monitors {u'b9cda581-f9ac-4650-83e0-17d00d7dfc75': <vdsm.storage.monitor.MonitorThread object at 0x3ecd6d0>, u'd4ad5a78-1d75-457b-8caf-400d2d7917a6': <vdsm.storage.monitor.MonitorThread object at 0x3e38890>, u'd6e4a622-bd31-4d8f-904d-1e26b7286757': <vdsm.storage.monitor.MonitorThread object at 0x3f2bf50>, u'6ffbc483-0031-403a-819b-3bb2f0f8de0a': <vdsm.storage.monitor.MonitorThread object at 0x3f2b5d0>, u'373e8c55-283f-41d4-8433-95c1ef1bbd1a': <vdsm.storage.monitor.MonitorThread object at 0x3ecde10>, u'2770b49f-79d7-4b7c-9d39-03e988e8bc4d': <vdsm.storage.monitor.MonitorThread object at 0x3f37690>}
irs._obj.domainMonitor._monitors['d6e4a622-bd31-4d8f-904d-1e26b7286757'].status.readDelay 0.000905336
irs._obj.domainMonitor._checker._loop._scheduled [<Timer when=4920799.730000 callback=<vdsm.storage.asyncutils.LoopingCall object at 0x2af6110> at 0x2396650>, <Timer when=4920799.750000 callback=<vdsm.storage.asyncutils.LoopingCall object at 0x2afc890> at 0x2396d50>, <Timer when=4920800.690000 callback=<vdsm.storage.asyncutils.LoopingCall object at 0x2b28890> at 0x2396590>, <Timer when=4920799.770000 callback=<vdsm.storage.asyncutils.LoopingCall object at 0x2b21c90> at 0x2331a90>, <Timer when=4920800.620000 callback=<vdsm.storage.asyncutils.LoopingCall object at 0x2b21d90> at 0x2a997d0>, <Timer when=4920800.800000 callback=<vdsm.storage.asyncutils.LoopingCall object at 0x2b9a9d0> at 0x23967d0>]
irs._obj.domainMonitor._checker._loop._scheduled[0]._callback._callback <bound method DirectioChecker._check of <DirectioChecker /rhev/data-center/mnt/dumbo.tlv.redhat.com:_voodoo_41/2770b49f-79d7-4b7c-9d39-03e988e8bc4d/dom_md/metadata running next_check=4920929.73 at 0x2af6050>>
dir(obj) is your best friend inside the manhole. Other ways to debug vdsm: - ps auxf - see vdsm child processes - htop - you can enable thread names and use tree mode to see which vdsm thread is consuming lot of cpu - strace - you can see which system calls vdsm is invoking - gdb - you can attach to vdsm and get a python backtrace using py-bt Nir
Have a nice day --------------------------------
pencc
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (2)
-
Nir Soffer -
pengyixiang