[ovirt-users] Ovirt host activation and lvm looping with high CPU load trying to mount iSCSI storage

[Nir Soffer]

On Tue, Jan 17, 2017 at 1:17 PM, Mark Greenall
<m.greenall at iontrading.com> wrote:
> Hi Nir,
>
> Thanks for your continuing efforts with this. It's really appreciated.
>
> re: number of storage domains, we'll plan a review of the underlying storage configuration to try and optimise things better. Do you have a recommended Storage Domain Size?
>
> Still running with the modified settings mentioned in my last mail. I'd like to try and work with yourselves to find a set of configuration variables we are all happy works with Ovirt and the Equallogic.
>
>>> 1. Please enable debug logging in sanlock log:
>>>
>>> edit /etc/sanlock/sanlock.conf and set:
>>> logfile_priority = 7
>
> Enabled, rebooted and tried another session this morning:
>
> 09:19 - Host Activated
> 09:21 - Non Operational (Cannot access the storage Domain Unknown)
> 09:25 - Connecting
> 09:31 - Manually rebooted host as getting nowhere.
> Throughout the above I see lots of LVM processes.
>
> I've attached all the logs for the above session. It may be that the stripped back settings are now causing another problem as this doesn't look like the same cycle I was previously seeing. I also see iSCSI connection errors in the messages file too.
>
> Multipath -ll shows a single path to the devices now rather than the two I previously had.
> Iscsiadm shows a single session for each domain as connected but spread between the Equallogic eth0 and eth1 interfaces. Previously I had two sessions for each domain, one connected to eth0 and one connected to eth1.
> Pvdisplay, vgdisplay and lvdisplay show a lot of LVM's and all seem to be 'available'

I guess you old configuration (other then then the new multipath.conf)
is better then the
defaults for now.

>
>>> 2. Try vdsm patch eliminating the delays in the monitoring thread
>
> When I check the gerrit link I see two monitor.py files (monitor_new.py.zip and storage_monitor_test_new.py.zip) which one should I be testing?

The best way is to use git, checkout the patch, and build new rpms.
git fetch https://gerrit.ovirt.org/vdsm refs/changes/50/70450/1 && git
checkout FETCH_HEAD

Please keep the old file and check that the diff between the old and new file
match what we see in gerrit.
diff -u /usr/share/vdsm/storage/monitor.py.old
/usr/share/vdsm/storage/monitor.py
https://gerrit.ovirt.org/#/c/70450/1/vdsm/storage/monitor.py

Nir