[ovirt-users] Memory treshold
vasily.lamykin
vasily.lamykin at MegaFon.ru
Tue Apr 4 09:33:52 UTC 2017
engine-config -s LogMaxPhysicalMemoryUsedThresholdInPercentage=%percent per threshold%
Василий Ламыкин
Старший инженер по эксплуатации сервисных платформ
Эксплуатация сети
Столичный филиал ПАО «МегаФон»
+7 (926) 500-3308
-----Original Message-----
From: users-bounces at ovirt.org [mailto:users-bounces at ovirt.org] On Behalf Of users-request at ovirt.org
Sent: Wednesday, March 29, 2017 1:00 PM
To: users at ovirt.org
Subject: Users Digest, Vol 66, Issue 254
Send Users mailing list submissions to
users at ovirt.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.ovirt.org/mailman/listinfo/users
or, via email, send a message with subject or body 'help' to
users-request at ovirt.org
You can reach the person managing the list at
users-owner at ovirt.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Users digest..."
Today's Topics:
1. Re: VM "hot" RAM memory decrease (Martin Sivak)
2. Memory treshold (vasily.lamykin)
3. Re: iSCSI Discovery cannot detetect LUN (Eduardo Mayoral)
----------------------------------------------------------------------
Message: 1
Date: Wed, 29 Mar 2017 11:50:59 +0200
From: Martin Sivak <msivak at redhat.com>
To: Nicol?s <nicolas at devels.es>
Cc: users <users at ovirt.org>
Subject: Re: [ovirt-users] VM "hot" RAM memory decrease
Message-ID:
<CAF0zDV5BG9PmzpaTS+73QKYELTv8ThNJcZoiGoySYsEuGV2c9Q at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Hi Nicolas,
you are right, the log shows exactly the right values. Those numbers are exactly what is sent to libvirt using currentMemory field in https://libvirt.org/formatdomain.html#elementsMemoryAllocation
So if the VM does not have enough memory, we will have to start looking there.
Best regards
Martin Sivak
On Wed, Mar 29, 2017 at 11:04 AM, <nicolas at devels.es> wrote:
> Hi Martin,
>
> I do, I'm attaching it. I merged the full day's log into one.
> Significant hours start at 11:24:57 (log time). The interesting
> machine is called jbaspre01, and as per what I can deduce, the values
> in log are correct (it never goes under the threshold of 1GB, which is
> its minimum). However, my colleagues and I are 100% sure we saw this
> machine having about 600K KiB when running top.
>
> This machine has:
> Memory size: 8196 MB
> Minimal guaranteed memory: 1024 MB
>
> If you need additional information don't hesitate to ask.
>
> Regards.
>
>
> El 2017-03-29 09:44, Martin Sivak escribi?:
>>
>> Hi Nicolas,
>>
>> KiB Mem tells you how much memory the host has. I am actually more
>> interested in the VM. Do you still have the mom.log from the time
>> this happened? We keep the logs for some time (they are rotated).
>>
>> Best regards
>>
>> Martin
>>
>> On Wed, Mar 29, 2017 at 10:28 AM, <nicolas at devels.es> wrote:
>>>
>>> El 2017-03-28 15:33, Martin Sivak escribi?:
>>>>>
>>>>>
>>>>> min guaranteed should be respected, adding mom maintainer Martin
>>>>
>>>>
>>>>
>>>> I can't really help without the mom.log and the virsh dumpxml
>>>> output of that VM in the final state (all memory ballooned).
>>>>
>>>> Min guaranteed should be respected, but the question is how the
>>>> physical memory was measured (which number from top was used).
>>>>
>>>> --
>>>> Martin Sivak
>>>> SLA / oVirt
>>>>
>>>
>>> Hi Martin,
>>>
>>> I tried to reproduce the issue again and I was unable to. I had a
>>> host with ~90% of RAM utilization and in mom.log I could see that
>>> ballooning was indeed happening, but I couldn't see a VM going under
>>> its minimum guaranteed threshold (at least not significantly).
>>>
>>> On top, I was using the 'KiB Mem' value to compare. I know this is
>>> in KiB and that oVirt has its values as MB in the webadmin, but when
>>> I sent this message the difference was quite more significant
>>> (1024MB of minimum guaranteed and about 640K KiB on the VM).
>>>
>>> It's also true that meanwhile we upgraded from 4.0.2 to 4.1.1 and
>>> this time I couldn't reproduce it anymore.
>>>
>>> If at some point I am able to reproduce it again, I'll send detailed
>>> logs of the issue.
>>>
>>> Thanks.
>>>
>>>
>>>> On Tue, Mar 28, 2017 at 4:29 PM, Michal Skrivanek
>>>> <michal.skrivanek at redhat.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> On 27 Mar 2017, at 23:15, Nicol?s <nicolas at devels.es> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I don't really know if this question is suitable on this list, as
>>>>>> I doubt it's an oVirt bug, neither I know if it shall be
>>>>>> considered a bug.
>>>>>>
>>>>>> We recently run a VM on a host that was memory over-used (around
>>>>>> 80% of usage). The VM booted correctly, then we run "top" and saw
>>>>>> how physical RAM started decreasing every two seconds. At first
>>>>>> it was 4GB, then less and less until it stabilized at around
>>>>>> 600MB.
>>>>>>
>>>>>> Based on this (correct me if I'm wrong), we believe this is an
>>>>>> effect of having this VM with ballooning enabled, as it does
>>>>>> exactly this: It adds/removes RAM depending on host decision.
>>>>>> Thing is that this VM had a minimal guaranteed RAM of 1GB, so
>>>>>> even if this happened due to ballooning, I'm not sure if it
>>>>>> should have honored the minimum guaranteed RAM.
>>>>>>
>>>>>> When this happened, we run a 'ps' to see with what options the
>>>>>> qemu process was invoked, and parameters seemed correct (that's
>>>>>> why I don't know if it should be posted here or even if it's a
>>>>>> bug).
>>>>>
>>>>>
>>>>>
>>>>> It does belong here, it?s a different component doing it ?externally?
>>>>> to
>>>>> the qemu process- mom.
>>>>>
>>>>>>
>>>>>> Is this the expected behavior?
>>>>>
>>>>>
>>>>>
>>>>> min guaranteed should be respected, adding mom maintainer Martin
>>>>>
>>>>> Thanks,
>>>>> michal
>>>>>
>>>>>>
>>>>>> Thanks.
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>
>>>>>>
>>>>>
>
------------------------------
Message: 2
Date: Wed, 29 Mar 2017 09:46:18 +0000
From: vasily.lamykin <vasily.lamykin at MegaFon.ru>
To: "users at ovirt.org" <users at ovirt.org>
Subject: [ovirt-users] Memory treshold
Message-ID: <78c01f9d8162434bbebbfe6a92222ba0 at vlg-ums05.Megafon.ru>
Content-Type: text/plain; charset="utf-8"
Hi all.
Explain please, where I can disable for cluster this threshold:
Used memory of host *** [98%] exceeded defined threshold [95%].
?
??????? ???????
??????? ??????? ?? ???????????? ????????? ????????
???????????? ????
????????? ?????? ??? ?????????
+7 (926) 500-3308
[??????? ????+???? ??? B2C]
________________________________
?????????? ? ???? ????????? ????????????? ????????????? ??? ?????????? ???, ??????? ??? ??????????. ? ????????? ????? ??????????? ???????????????? ??????????, ??????? ?? ????? ???? ???????? ??? ???????????? ???-????, ????? ?????????. ???? ?? ?? ??????? ????? ?????????, ?? ?????????????, ?????????????, ??????????? ??? ??????????????? ?????????? ????????? ??? ??? ????? ????????? ? ?????????. ???? ?? ???????? ??? ????????? ????????, ??????????, ??????????????? ???????? ??????????? ?? ???? ? ??????? ?? ???? ?????????? ???? ????????? ? ????? ????????? ??? ????? ? ??????????.
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. The contents may not be disclosed or used by anyone other than the addressee. If you are not the intended recipient(s), any use, disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you have received this communication in error please notify us immediately by responding to this email and then delete the e-mail and all attachments and any copies thereof.
(c)20mf50
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170329/5be75efd/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 2130 bytes
Desc: image001.png
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170329/5be75efd/attachment-0001.png>
------------------------------
Message: 3
Date: Wed, 29 Mar 2017 11:59:46 +0200
From: Eduardo Mayoral <emayoral at arsys.es>
To: Luk?? Kaplan <lkaplan at dragon.cz>,users <users at ovirt.org>
Subject: Re: [ovirt-users] iSCSI Discovery cannot detetect LUN
Message-ID: <2e73ca85-6e54-2994-b25f-4a11a03baac4 at arsys.es>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
I had a similar problem, in my case this was related to multipath, it was not masking the LUNs correctly, it was seeing it multiple times (one per path), and I could not select the LUNs in the oVirt interface.
Once I configured multipath correctly, everything worked like a charm.
Best regards,
--
Eduardo Mayoral.
On 29/03/17 11:30, Luk?? Kaplan wrote:
> Hello all,
>
> I did all steps as I described in previous email, but no change. I
> can't see any LUN after discovery and login of new iSCSI storage.
> (That storage is ok, if I try to connect it to another and older ovirt
> domain, it is working...)
>
> I tryed it on 3 new iSCSI targets alredy, all have same problem...
>
> Can somebody help me, please?
>
> --
> Lukas Kaplan
>
>
> 2017-03-27 16:22 GMT+02:00 Luk?? Kaplan <lkaplan at dragon.cz
> <mailto:lkaplan at dragon.cz>>:
>
> I did following steps:
> - delete target on all initiators (ovirt nodes)
> iscsiadm -m node -T iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T -p
> 10.53.1.201:3260 <http://10.53.1.201:3260> -u
> iscsiadm -m node -T iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T -p
> 10.53.1.201:3260 <http://10.53.1.201:3260> -o delete
> - stop tgtd on target
> - fill storage by zeroes (dd if=/dev/zero of=/dev/md125 bs=4096
> status=progress)
> - start tgtd
> - tried to connect to ovirt (Discovery=ok, Login=ok, but can not
> see any LUN).
>
> === After that I ran this commands on one node: ===
>
> [root at fudi-cn1 ~]# iscsiadm -m session -o show
> tcp: [1] 10.53.0.10:3260 <http://10.53.0.10:3260>,1
> iqn.2017-03.cz.dragon.ovirt:ovirtengine (non-flash)
> tcp: [11] 10.53.0.201:3260 <http://10.53.0.201:3260>,1
> iqn.2017-03.cz.dragon.ovirt.fudi-sn1:10T (non-flash)
> tcp: [12] 10.53.1.201:3260 <http://10.53.1.201:3260>,1
> iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T (non-flash)
>
> [root at fudi-cn1 ~]# iscsiadm -m discoverydb -P1
> SENDTARGETS:
> DiscoveryAddress: 10.53.0.201,3260
> Target: iqn.2017-03.cz.dragon.ovirt:ovirtengine
> Portal: 10.53.0.201:3260 <http://10.53.0.201:3260>,1
> Iface Name: default
> iSNS:
> No targets found.
> STATIC:
> Target: iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T
> Portal: 10.53.1.201:3260 <http://10.53.1.201:3260>,1
> Iface Name: default
> Target: iqn.2017-03.cz.dragon.ovirt:ovirtengine
> Portal: 10.53.0.10:3260 <http://10.53.0.10:3260>,1
> Iface Name: default
> Target: iqn.2017-03.cz.dragon.ovirt.fudi-sn1:10T
> Portal: 10.53.0.201:3260 <http://10.53.0.201:3260>,1
> Iface Name: default
> FIRMWARE:
> No targets found.
>
> === On iscsi target: ===
> [root at fuvs-sn1 ~]# cat /proc/mdstat
> Personalities : [raid1] [raid6] [raid5] [raid4]
> md125 : active raid6 sdl1[11] sdk1[10] sdj1[9] sdi1[8] sdh1[7]
> sdg1[6] sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1] sda1[0]
> 9766302720 blocks super 1.2 level 6, 512k chunk, algorithm 2
> [12/12] [UUUUUUUUUUUU]
> bitmap: 0/8 pages [0KB], 65536KB chunk
> ...etc...
>
>
> [root at fuvs-sn1 ~]# cat /etc/tgt/targets.conf
> default-driver iscsi
>
> <target iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T>
> # provided devicce as a iSCSI target
> backing-store /dev/md125
> # iSCSI Initiator's IP address you allow to connect
> #initiator-address 10.53.0.0/23 <http://10.53.0.0/23>
> </target>
>
> --
> Lukas Kaplan
>
> 2017-03-25 12:36 GMT+01:00 Lukas Kaplan <lkaplan at dragon.cz
> <mailto:lkaplan at dragon.cz>>:
>
> Co muze myslet tim mappingem?
>
> Jinak muzu zkusit ddckem celou storage prepsat nulami.
>
> co ty na to?
>
> Odesl?no z iPhonu
>
> Za??tek p?epos?lan? zpr?vy:
>
>> *Od:* Yaniv Kaul <ykaul at redhat.com <mailto:ykaul at redhat.com>>
>> *Datum:* 24. b?ezna 2017 23:25:21 SE?
>> *Komu:* Luk?? Kaplan <lkaplan at dragon.cz
>> <mailto:lkaplan at dragon.cz>>
>> *Kopie:* users <users at ovirt.org <mailto:users at ovirt.org>>
>> *P?edm?t:* *Re: [ovirt-users] iSCSI Discovery cannot detetect
>> LUN*
>>
>>
>>
>> On Fri, Mar 24, 2017 at 1:34 PM, Luk?? Kaplan
>> <lkaplan at dragon.cz <mailto:lkaplan at dragon.cz>> wrote:
>>
>> Hello all,
>>
>> please do you have some experience with troubleshooting
>> adding of iSCSI domain to ovirt 4.1.1?
>>
>> I am chalenging this issue now:
>>
>> 1) I have successfuly installed oVirt 4.1.1 environment
>> with self-hosted engine, 3 nodes and 3 storages (iSCSI
>> Master domain, iSCSI for hosted engine and NFS ISO
>> domain). Everything is working now.
>>
>> 2) But, when I want to add new iSCSI domain, I can
>> discover it, I can login, but I cant see any LUN on that
>> storage. (I had same problem in oVirt 4.1.0, so I made
>> upgrade to 4.1.1)
>>
>>
>> Are you sure mappings are correct?
>> Can you ensure the LUN is empty?
>> Y.
>>
>>
>> 3) Then I tryed to add this storage to another oVirt
>> environment (oVirt 3.6) and there are no problem. I can
>> see LUN on that storage and I can connect it to oVirt.
>>
>> I tryed to examine vdsm.log, but it is very detailed and
>> unredable for me :-/
>>
>> Thak you in advance, have a nice day,
>> --
>> Lukas Kaplan
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org <mailto:Users at ovirt.org>
>> http://lists.ovirt.org/mailman/listinfo/users
>> <http://lists.ovirt.org/mailman/listinfo/users>
>>
>>
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170329/e73ab86f/attachment.html>
------------------------------
_______________________________________________
Users mailing list
Users at ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
End of Users Digest, Vol 66, Issue 254
**************************************
________________________________
Информация в этом сообщении предназначена исключительно для конкретных лиц, которым она адресована. В сообщении может содержаться конфиденциальная информация, которая не может быть раскрыта или использована кем-либо, кроме адресатов. Если вы не адресат этого сообщения, то использование, переадресация, копирование или распространение содержания сообщения или его части незаконно и запрещено. Если Вы получили это сообщение ошибочно, пожалуйста, незамедлительно сообщите отправителю об этом и удалите со всем содержимым само сообщение и любые возможные его копии и приложения.
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. The contents may not be disclosed or used by anyone other than the addressee. If you are not the intended recipient(s), any use, disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you have received this communication in error please notify us immediately by responding to this email and then delete the e-mail and all attachments and any copies thereof.
(c)20mf50
More information about the Users
mailing list