----- Original Message -----

From: "Francesco Romani" <fromani@redhat.com> To: devel@ovirt.org Sent: Friday, July 4, 2014 12:09:44 PM Subject: [ovirt-devel] [VDSM] VDSM threads as in vdsm 4.15.0

Hi,

I'm trying to enumerate all the daemon threads VDSM uses, as in 4.15.0 (to be exact, as in master 3228f5dad8f488192ea62f4e3e39172832435e80)

So far I got:

===

SocketServer: 1 service, short lived thread per request?

VDSM alone, with no VM running

num what where -------------------------------------------------------- 1 libVirtEventLoop lib/vdsm/libvirtconnection.py P TaskManager vdsm/storage/hsm.py 1 StorageServer vdsm/storage/storageServer.py 1 StorageRefresh vdsm/storage/hsm.py 1 ChannelListener vdsm/virt/vmchannels.py 1 HostStatsThread vdsm/virt/sampling.py 1 MomThread vdsm/momIF.py 1 clientIF vdsm/clientIF.py 1 HostMonitor mom/HostMonitor.py 1 StompReactor vdsm/rpc/BindingJsonRpc.py 1 JsonRpcServer vdsm/rpc/BidningJsonRpc.py 1 GuestManager mom/GuestManager.py 1 PolicyEngine mom/PolicyEngine.py 1 XmlRpcServer? vdsm/rpc/BindingXMLRPC.py 1 Detector vdsm/clientIF.py 1 RPCServer mom/RPCServer.py 1 SocketServer stdlib, triggered by vdsm/rpc/BindingXMLRPC.py 1 IOProcess? vdsm/storage/outOfProcess.py S DomainMonitor vdsm/storage/domainMonitor.py P/2 HSM_MailMonitor vdsm/storage/storage_mailbox.py P/2 SPM_MailMonitor vdsm/storage/storage_mailbox.py

P = pool size (config/irs/thread_pool_size)

Total: 17 + number(StorageDomains) + P * 2 Total with defaults (P=10) = 37 + number(StorageDomains) Minimum storage domains: 2 (ISO + data) Baseline: 39 threads

Per VM 1 sampling vdsm/virt/sampling.py 1 GuestMonitor mom/GuestMonitor.py

===

More precise update will follow. In the meantime, anyone please feel free to point out omissions/errors.

Aren't you forgetting those of supervdsmd?

Thanks,

-- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

----- Original Message -----

From: "Francesco Romani" <fromani@redhat.com> To: devel@ovirt.org Sent: Friday, July 4, 2014 1:09:44 PM Subject: [ovirt-devel] [VDSM] VDSM threads as in vdsm 4.15.0

Hi,

I'm trying to enumerate all the daemon threads VDSM uses, as in 4.15.0 (to be exact, as in master 3228f5dad8f488192ea62f4e3e39172832435e80)

So far I got:

===

SocketServer: 1 service, short lived thread per request?

We don't have such thread, nor short lived threads per requests (but see bellow)

VDSM alone, with no VM running

num what where -------------------------------------------------------- 1 libVirtEventLoop lib/vdsm/libvirtconnection.py P TaskManager vdsm/storage/hsm.py

This is not a thread - you mean the pool of threadPool.WorkerThread

1 StorageServer vdsm/storage/storageServer.py

I guess you mean storageServer.ConnectionMonitor thread

1 StorageRefresh vdsm/storage/hsm.py

This thread lives only during startup and does not exists when vdsm is ready.

1 ChannelListener vdsm/virt/vmchannels.py 1 HostStatsThread vdsm/virt/sampling.py 1 MomThread vdsm/momIF.py

There is also ksm.KsmMonitorThread

1 clientIF vdsm/clientIF.py

This is not a thread, don't let the interface confuse you :-) The start() method starts the bindings threads and the protocol detector you list bellow. This is another short lived thread the exit when running vms are recovered and does not exists when vdsm is ready.

1 HostMonitor mom/HostMonitor.py 1 StompReactor vdsm/rpc/BindingJsonRpc.py 1 JsonRpcServer vdsm/rpc/BidningJsonRpc.py

This thread create short living threads for each request, without any limit. If you send 100 requests, vdsm will start 100 threads the handle the requests. (since Jun 11)

1 GuestManager mom/GuestManager.py 1 PolicyEngine mom/PolicyEngine.py 1 XmlRpcServer? vdsm/rpc/BindingXMLRPC.py

This server create new thread per connection. Since ovirt-3.4. This thread is long living since we use keep-alive (since 3.4). There is no limit to the number of connection threads. On and idle machine we see 3-4 such threads. So you should add 4 to the baseline for these.

1 Detector vdsm/clientIF.py 1 RPCServer mom/RPCServer.py 1 SocketServer stdlib, triggered by vdsm/rpc/BindingXMLRPC.py

There is no such thread - you counted it in XmlRpcServer.

1 IOProcess? vdsm/storage/outOfProcess.py S DomainMonitor vdsm/storage/domainMonitor.py

When domain monitor changes state, it starts a new thread for notifying observers and this thread may start new thread for each notification. These thread exit after the notifications are done.

P/2 HSM_MailMonitor vdsm/storage/storage_mailbox.py

These run on all hosts but the spm

P/2 SPM_MailMonitor vdsm/storage/storage_mailbox.py

These run only on the spm - so we have only pool_size / 2 mailbox threads.

P = pool size (config/irs/thread_pool_size)

Total: 17 + number(StorageDomains) + P * 2 Total with defaults (P=10) = 37 + number(StorageDomains) Minimum storage domains: 2 (ISO + data)

You don't need iso domain, so minimum is 1 storage domain thread. But the minimum is not interesting - we have customers with 30-40 domains, so assume 40 of these threads.

Baseline: 39 threads

Based on my corrections: 14 long living threads 10 task threads 5 mailbox threads 3-? xmlprc connection threads or ? jsonrpc requests threads 1-40 domain monitor threads 0-600 vm threads (2 per vm) Total: 33-? Nir