5:09 a.m.
Hi,
I'm trying to enumerate all the daemon threads VDSM uses, as in 4.15.0
(to be exact, as in master 3228f5dad8f488192ea62f4e3e39172832435e80)
So far I got:
===
SocketServer: 1 service, short lived thread per request?
VDSM alone, with no VM running
num what where
--------------------------------------------------------
1 libVirtEventLoop lib/vdsm/libvirtconnection.py
P TaskManager vdsm/storage/hsm.py
1 StorageServer vdsm/storage/storageServer.py
1 StorageRefresh vdsm/storage/hsm.py
1 ChannelListener vdsm/virt/vmchannels.py
1 HostStatsThread vdsm/virt/sampling.py
1 MomThread vdsm/momIF.py
1 clientIF vdsm/clientIF.py
1 HostMonitor mom/HostMonitor.py
1 StompReactor vdsm/rpc/BindingJsonRpc.py
1 JsonRpcServer vdsm/rpc/BidningJsonRpc.py
1 GuestManager mom/GuestManager.py
1 PolicyEngine mom/PolicyEngine.py
1 XmlRpcServer? vdsm/rpc/BindingXMLRPC.py
1 Detector vdsm/clientIF.py
1 RPCServer mom/RPCServer.py
1 SocketServer stdlib, triggered by vdsm/rpc/BindingXMLRPC.py
1 IOProcess? vdsm/storage/outOfProcess.py
S DomainMonitor vdsm/storage/domainMonitor.py
P/2 HSM_MailMonitor vdsm/storage/storage_mailbox.py
P/2 SPM_MailMonitor vdsm/storage/storage_mailbox.py
P = pool size (config/irs/thread_pool_size)
Total: 17 + number(StorageDomains) + P * 2
Total with defaults (P=10) = 37 + number(StorageDomains)
Minimum storage domains: 2 (ISO + data)
Baseline: 39 threads
Per VM
1 sampling vdsm/virt/sampling.py
1 GuestMonitor mom/GuestMonitor.py
===
More precise update will follow. In the meantime, anyone please feel free to
point out omissions/errors.
Thanks,
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
8:10 a.m.
----- Original Message -----
From: "Francesco Romani" <fromani(a)redhat.com>
To: devel(a)ovirt.org
Sent: Friday, July 4, 2014 12:09:44 PM
Subject: [ovirt-devel] [VDSM] VDSM threads as in vdsm 4.15.0
Hi,
I'm trying to enumerate all the daemon threads VDSM uses, as in 4.15.0
(to be exact, as in master 3228f5dad8f488192ea62f4e3e39172832435e80)
So far I got:
===
SocketServer: 1 service, short lived thread per request?
VDSM alone, with no VM running
num what where
--------------------------------------------------------
1 libVirtEventLoop lib/vdsm/libvirtconnection.py
P TaskManager vdsm/storage/hsm.py
1 StorageServer vdsm/storage/storageServer.py
1 StorageRefresh vdsm/storage/hsm.py
1 ChannelListener vdsm/virt/vmchannels.py
1 HostStatsThread vdsm/virt/sampling.py
1 MomThread vdsm/momIF.py
1 clientIF vdsm/clientIF.py
1 HostMonitor mom/HostMonitor.py
1 StompReactor vdsm/rpc/BindingJsonRpc.py
1 JsonRpcServer vdsm/rpc/BidningJsonRpc.py
1 GuestManager mom/GuestManager.py
1 PolicyEngine mom/PolicyEngine.py
1 XmlRpcServer? vdsm/rpc/BindingXMLRPC.py
1 Detector vdsm/clientIF.py
1 RPCServer mom/RPCServer.py
1 SocketServer stdlib, triggered by vdsm/rpc/BindingXMLRPC.py
1 IOProcess? vdsm/storage/outOfProcess.py
S DomainMonitor vdsm/storage/domainMonitor.py
P/2 HSM_MailMonitor vdsm/storage/storage_mailbox.py
P/2 SPM_MailMonitor vdsm/storage/storage_mailbox.py
P = pool size (config/irs/thread_pool_size)
Total: 17 + number(StorageDomains) + P * 2
Total with defaults (P=10) = 37 + number(StorageDomains)
Minimum storage domains: 2 (ISO + data)
Baseline: 39 threads
Per VM
1 sampling vdsm/virt/sampling.py
1 GuestMonitor mom/GuestMonitor.py
===
More precise update will follow. In the meantime, anyone please feel free to
point out omissions/errors.
Aren't you forgetting those of supervdsmd?
Thanks,
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
9:14 a.m.
On Fri, Jul 04, 2014 at 06:09:44AM -0400, Francesco Romani wrote:
Hi,
I'm trying to enumerate all the daemon threads VDSM uses, as in 4.15.0
(to be exact, as in master 3228f5dad8f488192ea62f4e3e39172832435e80)
So far I got:
===
SocketServer: 1 service, short lived thread per request?
VDSM alone, with no VM running
num what where
--------------------------------------------------------
1 libVirtEventLoop lib/vdsm/libvirtconnection.py
P TaskManager vdsm/storage/hsm.py
1 StorageServer vdsm/storage/storageServer.py
1 StorageRefresh vdsm/storage/hsm.py
1 ChannelListener vdsm/virt/vmchannels.py
1 HostStatsThread vdsm/virt/sampling.py
1 MomThread vdsm/momIF.py
1 clientIF vdsm/clientIF.py
1 HostMonitor mom/HostMonitor.py
1 StompReactor vdsm/rpc/BindingJsonRpc.py
1 JsonRpcServer vdsm/rpc/BidningJsonRpc.py
1 GuestManager mom/GuestManager.py
1 PolicyEngine mom/PolicyEngine.py
1 XmlRpcServer? vdsm/rpc/BindingXMLRPC.py
1 Detector vdsm/clientIF.py
1 RPCServer mom/RPCServer.py
1 SocketServer stdlib, triggered by vdsm/rpc/BindingXMLRPC.py
1 IOProcess? vdsm/storage/outOfProcess.py
S DomainMonitor vdsm/storage/domainMonitor.py
P/2 HSM_MailMonitor vdsm/storage/storage_mailbox.py
P/2 SPM_MailMonitor vdsm/storage/storage_mailbox.py
P = pool size (config/irs/thread_pool_size)
Total: 17 + number(StorageDomains) + P * 2
Total with defaults (P=10) = 37 + number(StorageDomains)
Minimum storage domains: 2 (ISO + data)
Baseline: 39 threads
Per VM
1 sampling vdsm/virt/sampling.py
1 GuestMonitor mom/GuestMonitor.py
===
More precise update will follow. In the meantime, anyone please feel free to
point out omissions/errors.
You must have dropped the migration-monitoring threads on purposed! (but
they still pop up). There are other ocasions when (hopefully)
short-lived threads are created, such as during getStorageDomainsList()
XmlRpcServer springs a new thread per http connection.
What's P/2 ? What are the question marks in the above table?
And in that line - what's the IOProcess thead?
Dan.
1:58 p.m.
----- Original Message -----
From: "Francesco Romani" <fromani(a)redhat.com>
To: devel(a)ovirt.org
Sent: Friday, July 4, 2014 1:09:44 PM
Subject: [ovirt-devel] [VDSM] VDSM threads as in vdsm 4.15.0
Hi,
I'm trying to enumerate all the daemon threads VDSM uses, as in 4.15.0
(to be exact, as in master 3228f5dad8f488192ea62f4e3e39172832435e80)
So far I got:
===
SocketServer: 1 service, short lived thread per request?
We don't have such thread, nor short lived threads per requests (but see bellow)
VDSM alone, with no VM running
num what where
--------------------------------------------------------
1 libVirtEventLoop lib/vdsm/libvirtconnection.py
P TaskManager vdsm/storage/hsm.py
This is not a thread - you mean the pool of threadPool.WorkerThread
1 StorageServer vdsm/storage/storageServer.py
I guess you mean storageServer.ConnectionMonitor thread
1 StorageRefresh vdsm/storage/hsm.py
This thread lives only during startup and does not exists when vdsm is ready.
1 ChannelListener vdsm/virt/vmchannels.py
1 HostStatsThread vdsm/virt/sampling.py
1 MomThread vdsm/momIF.py
There is also ksm.KsmMonitorThread
1 clientIF vdsm/clientIF.py
This is not a thread, don't let the interface confuse you :-)
The start() method starts the bindings threads and the protocol detector
you list bellow.
This is another short lived thread the exit when running vms are recovered
and does not exists when vdsm is ready.
1 HostMonitor mom/HostMonitor.py
1 StompReactor vdsm/rpc/BindingJsonRpc.py
1 JsonRpcServer vdsm/rpc/BidningJsonRpc.py
This thread create short living threads for each request, without any limit.
If you send 100 requests, vdsm will start 100 threads the handle the requests.
(since Jun 11)
1 GuestManager mom/GuestManager.py
1 PolicyEngine mom/PolicyEngine.py
1 XmlRpcServer? vdsm/rpc/BindingXMLRPC.py
This server create new thread per connection. Since ovirt-3.4. This thread
is long living since we use keep-alive (since 3.4).
There is no limit to the number of connection threads. On and idle machine
we see 3-4 such threads. So you should add 4 to the baseline for these.
1 Detector vdsm/clientIF.py
1 RPCServer mom/RPCServer.py
1 SocketServer stdlib, triggered by vdsm/rpc/BindingXMLRPC.py
There is no such thread - you counted it in XmlRpcServer.
1 IOProcess? vdsm/storage/outOfProcess.py
S DomainMonitor vdsm/storage/domainMonitor.py
When domain monitor changes state, it starts a new thread for notifying
observers and this thread may start new thread for each notification.
These thread exit after the notifications are done.
P/2 HSM_MailMonitor vdsm/storage/storage_mailbox.py
These run on all hosts but the spm
P/2 SPM_MailMonitor vdsm/storage/storage_mailbox.py
These run only on the spm - so we have only pool_size / 2 mailbox threads.
P = pool size (config/irs/thread_pool_size)
Total: 17 + number(StorageDomains) + P * 2
Total with defaults (P=10) = 37 + number(StorageDomains)
Minimum storage domains: 2 (ISO + data)
You don't need iso domain, so minimum is 1 storage domain thread.
But the minimum is not interesting - we have customers with 30-40 domains,
so assume 40 of these threads.
Baseline: 39 threads
Based on my corrections:
14 long living threads
10 task threads
5 mailbox threads
3-? xmlprc connection threads or ? jsonrpc requests threads
1-40 domain monitor threads
0-600 vm threads (2 per vm)
Total: 33-?
Nir
7:10 a.m.
----- Original Message -----
From: "Francesco Romani" <fromani(a)redhat.com>
To: devel(a)ovirt.org
Sent: Friday, July 4, 2014 1:09:44 PM
Subject: [ovirt-devel] [VDSM] VDSM threads as in vdsm 4.15.0
Hi,
I'm trying to enumerate all the daemon threads VDSM uses, as in 4.15.0
(to be exact, as in master 3228f5dad8f488192ea62f4e3e39172832435e80)
So far I got:
===
SocketServer: 1 service, short lived thread per request?
VDSM alone, with no VM running
num what where
--------------------------------------------------------
1 libVirtEventLoop lib/vdsm/libvirtconnection.py
P TaskManager vdsm/storage/hsm.py
1 StorageServer vdsm/storage/storageServer.py
1 StorageRefresh vdsm/storage/hsm.py
1 ChannelListener vdsm/virt/vmchannels.py
1 HostStatsThread vdsm/virt/sampling.py
1 MomThread vdsm/momIF.py
1 clientIF vdsm/clientIF.py
1 HostMonitor mom/HostMonitor.py
1 StompReactor vdsm/rpc/BindingJsonRpc.py
1 JsonRpcServer vdsm/rpc/BidningJsonRpc.py
Will spawn one thread per
request (to keep the same
behavior as the XMLRPC server. There is a parameter
in the constructor to set an alternative scheduler.)
1 GuestManager mom/GuestManager.py
1 PolicyEngine mom/PolicyEngine.py
1 XmlRpcServer? vdsm/rpc/BindingXMLRPC.py
1 Detector vdsm/clientIF.py
1 RPCServer mom/RPCServer.py
1 SocketServer stdlib, triggered by vdsm/rpc/BindingXMLRPC.py
1 IOProcess? vdsm/storage/outOfProcess.py
Currently one per IOProcess
slave. It's for log collection.
We could consolidate it to one for all IOProcess instances if
it ever becomes an issue. There is currently only one global
IOProcess but we want to change it so there is an IOProcess instance
per domain.
S DomainMonitor vdsm/storage/domainMonitor.py
P/2 HSM_MailMonitor vdsm/storage/storage_mailbox.py
P/2 SPM_MailMonitor vdsm/storage/storage_mailbox.py
P = pool size (config/irs/thread_pool_size)
Total: 17 + number(StorageDomains) + P * 2
Total with defaults (P=10) = 37 + number(StorageDomains)
Minimum storage domains: 2 (ISO + data)
Baseline: 39 threads
Per VM
1 sampling vdsm/virt/sampling.py
1 GuestMonitor mom/GuestMonitor.py
===
More precise update will follow. In the meantime, anyone please feel free to
point out omissions/errors.
Thanks,
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
4006
days inactive
4008
days old
4 comments
5 participants
participants (5)
-
Antoni Segura Puimedon -
Dan Kenigsberg -
Francesco Romani -
Nir Soffer -
Saggi Mizrahi