<div dir="ltr">On Sun, Nov 29, 2015 at 6:01 PM, Yaniv Kaul <<a href="mailto:ykaul@redhat.com">ykaul@redhat.com</a>> wrote:<br>> On Sun, Nov 29, 2015 at 5:37 PM, Nir Soffer <<a href="mailto:nsoffer@redhat.com">nsoffer@redhat.com</a>> wrote:<br>>><br>>> On Sun, Nov 29, 2015 at 10:37 AM, Yaniv Kaul <<a href="mailto:ykaul@redhat.com">ykaul@redhat.com</a>> wrote:<br>>> ><br>>> > On Fri, Nov 27, 2015 at 6:55 PM, Francesco Romani <<a href="mailto:fromani@redhat.com">fromani@redhat.com</a>><br>>> > wrote:<br>>> >><br>>> >> Using taskset, the ip command now takes a little longer to complete.<br>>> ><br>>> ><br>>> > Since we always use the same set of CPUs, I assume using a mask (for 0 &<br>>> > 1,<br>>> > just use 0x3, as the man suggests) might be a tiny of a fraction faster<br>>> > to<br>>> > execute taskset with, instead of the need to translate the numeric CPU<br>>> > list.<br>>><br>>> Creating the string "0-<last cpu index>" is one line in vdsm. The code<br>>> handling this in<br>>> taskset is written in C, so the parsing time is practically zero. Even<br>>> if it was non-zero,<br>>> this code run once when we run a child process, so the cost is<br>>> insignificant.<br>><br>><br>> I think it's easier to just to have it as a mask in a config item somewhere,<br>> without need to create it or parse it anywhere.<br>> For us and for the user.<br><br>We have this option in /etc/vdsm/vdsm.conf:<br><br><div><font face="monospace, monospace"> # Comma separated whitelist of CPU cores on which VDSM is allowed to</font></div><div><font face="monospace, monospace"> # run. The default is "", meaning VDSM can be scheduled by the OS to</font></div><div><font face="monospace, monospace"> # run on any core. Valid examples: "1", "0,1", "0,2,3"</font></div><div><font face="monospace, monospace"> # cpu_affinity = 1</font></div><div><br></div>I think this is the easiest option for users.<br><br>>> > However, the real concern is making sure CPUs 0 & 1 are not really too<br>>> > busy<br>>> > with stuff (including interrupt handling, etc.)<br>>><br>>> This code is used when we run a child process, to allow the child<br>>> process to run on<br>>> all cpus (in this case, cpu 0 and cpu 1). So I think there is no concern<br>>> here.<br>>><br>>> Vdsm itself is running by default on cpu 1, which should be less busy<br>>> then cpu 0.<br>><br>><br>> I assume those are cores, which probably in a multi-socket will be in the<br>> first socket only.<br>> There's a good chance that the FC and or network/cards will also bind their<br>> interrupts to core0 & core 1 (check /proc/interrupts) on the same socket.<br>> From my poor laptop (1s, 4c):<br>> 42: 1487104 9329 4042 3598 IR-PCI-MSI 512000-edge <br>> 0000:00:1f.2<br>><br>> (my SATA controller)<br>><br>> 43: 14664923 34 18 13 IR-PCI-MSI 327680-edge <br>> xhci_hcd<br>> (my dock station connector)<br>><br>> 45: 6754579 4437 2501 2419 IR-PCI-MSI 32768-edge <br>> i915<br>> (GPU)<br>><br>> 47: 187409 11627 1235 1259 IR-PCI-MSI 2097152-edge <br>> iwlwifi<br>> (NIC, wifi)<br><br>Interesting, here an example from a 8 cores machine running my vms:<div><br></div><div><div><font face="monospace, monospace">[nsoffer@jumbo ~]$ cat /proc/interrupts </font></div><div><font face="monospace, monospace"> CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 </font></div><div><font face="monospace, monospace"> 0: 31 0 0 0 0 0 0 0 IR-IO-APIC-edge timer</font></div><div><font face="monospace, monospace"> 1: 2 0 0 1 0 0 0 0 IR-IO-APIC-edge i8042</font></div><div><font face="monospace, monospace"> 8: 0 0 0 0 0 0 0 1 IR-IO-APIC-edge rtc0</font></div><div><font face="monospace, monospace"> 9: 0 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi acpi</font></div><div><font face="monospace, monospace"> 12: 3 0 0 0 0 0 1 0 IR-IO-APIC-edge i8042</font></div><div><font face="monospace, monospace"> 16: 4 4 9 0 9 1 1 3 IR-IO-APIC 16-fasteoi ehci_hcd:usb3</font></div><div><font face="monospace, monospace"> 23: 13 1 5 0 12 1 1 0 IR-IO-APIC 23-fasteoi ehci_hcd:usb4</font></div><div><font face="monospace, monospace"> 24: 0 0 0 0 0 0 0 0 DMAR_MSI-edge dmar0</font></div><div><font face="monospace, monospace"> 25: 0 0 0 0 0 0 0 0 DMAR_MSI-edge dmar1</font></div><div><font face="monospace, monospace"> 26: 3670 354 215 9062370 491 124 169 54 IR-PCI-MSI-edge 0000:00:1f.2</font></div><div><font face="monospace, monospace"> 27: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge xhci_hcd</font></div><div><font face="monospace, monospace"> 28: 166285414 0 3 0 4 0 0 0 IR-PCI-MSI-edge em1</font></div><div><font face="monospace, monospace"> 29: 18 0 0 0 4 3 0 0 IR-PCI-MSI-edge mei_me</font></div><div><font face="monospace, monospace"> 30: 1 151 17 0 3 169 26 94 IR-PCI-MSI-edge snd_hda_intel</font></div><div><font face="monospace, monospace">NMI: 2508 2296 2317 2356 867 918 912 903 Non-maskable interrupts</font></div><div><font face="monospace, monospace">LOC: 302996116 312923350 312295375 312089303 86282447 94046427 90847792 91761277 Local timer interrupts</font></div><div><font face="monospace, monospace">SPU: 0 0 0 0 0 0 0 0 Spurious interrupts</font></div><div><font face="monospace, monospace">PMI: 2508 2296 2317 2356 867 918 912 903 Performance monitoring interrupts</font></div><div><font face="monospace, monospace">IWI: 1 0 0 5 0 0 0 0 IRQ work interrupts</font></div><div><font face="monospace, monospace">RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries</font></div><div><font face="monospace, monospace">RES: 34480637 12953645 13139863 14309885 8881861 10110753 9709070 9703933 Rescheduling interrupts</font></div><div><font face="monospace, monospace">CAL: 7387779 7682087 7283716 7135792 2771105 1785528 1887493 1843734 Function call interrupts</font></div><div><font face="monospace, monospace">TLB: 11121 16458 17923 15216 8534 8173 8639 7837 TLB shootdowns</font></div><div><font face="monospace, monospace">TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts</font></div><div><font face="monospace, monospace">THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts</font></div><div><font face="monospace, monospace">MCE: 0 0 0 0 0 0 0 0 Machine check exceptions</font></div><div><font face="monospace, monospace">MCP: 7789 7789 7789 7789 7789 7789 7789 7789 Machine check polls</font></div><div><font face="monospace, monospace">HYP: 0 0 0 0 0 0 0 0 Hypervisor callback interrupts</font></div><div><font face="monospace, monospace">ERR: 0</font></div><div><font face="monospace, monospace">MIS: 0</font></div><div><br></div>It seems that our default (CPU1) is fine.<div><br></div><div>Francesco, what do you think?</div><div><br></div><div>Nir</div></div></div>