12:33 p.m.
engine-config -s LogMaxPhysicalMemoryUsedThresholdInPercentage=%percent per threshold%
Василий Ламыкин
Старший инженер по эксплуатации сервисных платформ
Эксплуатация сети
Столичный филиал ПАО «МегаФон»
+7 (926) 500-3308
-----Original Message-----
From: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] On Behalf Of
users-request(a)ovirt.org
Sent: Wednesday, March 29, 2017 1:00 PM
To: users(a)ovirt.org
Subject: Users Digest, Vol 66, Issue 254
Send Users mailing list submissions to
users(a)ovirt.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.ovirt.org/mailman/listinfo/users
or, via email, send a message with subject or body 'help' to
users-request(a)ovirt.org
You can reach the person managing the list at
users-owner(a)ovirt.org
When replying, please edit your Subject line so it is more specific than "Re:
Contents of Users digest..."
Today's Topics:
1. Re: VM "hot" RAM memory decrease (Martin Sivak)
2. Memory treshold (vasily.lamykin)
3. Re: iSCSI Discovery cannot detetect LUN (Eduardo Mayoral)
----------------------------------------------------------------------
Message: 1
Date: Wed, 29 Mar 2017 11:50:59 +0200
From: Martin Sivak <msivak(a)redhat.com>
To: Nicol?s <nicolas(a)devels.es>
Cc: users <users(a)ovirt.org>
Subject: Re: [ovirt-users] VM "hot" RAM memory decrease
Message-ID:
<CAF0zDV5BG9PmzpaTS+73QKYELTv8ThNJcZoiGoySYsEuGV2c9Q(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Hi Nicolas,
you are right, the log shows exactly the right values. Those numbers are exactly what is
sent to libvirt using currentMemory field in
https://libvirt.org/formatdomain.html#elementsMemoryAllocation
So if the VM does not have enough memory, we will have to start looking there.
Best regards
Martin Sivak
On Wed, Mar 29, 2017 at 11:04 AM, <nicolas(a)devels.es> wrote:
Hi Martin,
I do, I'm attaching it. I merged the full day's log into one.
Significant hours start at 11:24:57 (log time). The interesting
machine is called jbaspre01, and as per what I can deduce, the values
in log are correct (it never goes under the threshold of 1GB, which is
its minimum). However, my colleagues and I are 100% sure we saw this
machine having about 600K KiB when running top.
This machine has:
Memory size: 8196 MB
Minimal guaranteed memory: 1024 MB
If you need additional information don't hesitate to ask.
Regards.
El 2017-03-29 09:44, Martin Sivak escribi?:
>
> Hi Nicolas,
>
> KiB Mem tells you how much memory the host has. I am actually more
> interested in the VM. Do you still have the mom.log from the time
> this happened? We keep the logs for some time (they are rotated).
>
> Best regards
>
> Martin
>
> On Wed, Mar 29, 2017 at 10:28 AM, <nicolas(a)devels.es> wrote:
>>
>> El 2017-03-28 15:33, Martin Sivak escribi?:
>>>>
>>>>
>>>> min guaranteed should be respected, adding mom maintainer Martin
>>>
>>>
>>>
>>> I can't really help without the mom.log and the virsh dumpxml
>>> output of that VM in the final state (all memory ballooned).
>>>
>>> Min guaranteed should be respected, but the question is how the
>>> physical memory was measured (which number from top was used).
>>>
>>> --
>>> Martin Sivak
>>> SLA / oVirt
>>>
>>
>> Hi Martin,
>>
>> I tried to reproduce the issue again and I was unable to. I had a
>> host with ~90% of RAM utilization and in mom.log I could see that
>> ballooning was indeed happening, but I couldn't see a VM going under
>> its minimum guaranteed threshold (at least not significantly).
>>
>> On top, I was using the 'KiB Mem' value to compare. I know this is
>> in KiB and that oVirt has its values as MB in the webadmin, but when
>> I sent this message the difference was quite more significant
>> (1024MB of minimum guaranteed and about 640K KiB on the VM).
>>
>> It's also true that meanwhile we upgraded from 4.0.2 to 4.1.1 and
>> this time I couldn't reproduce it anymore.
>>
>> If at some point I am able to reproduce it again, I'll send detailed
>> logs of the issue.
>>
>> Thanks.
>>
>>
>>> On Tue, Mar 28, 2017 at 4:29 PM, Michal Skrivanek
>>> <michal.skrivanek(a)redhat.com> wrote:
>>>>
>>>>
>>>>
>>>>> On 27 Mar 2017, at 23:15, Nicol?s <nicolas(a)devels.es> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I don't really know if this question is suitable on this list,
as
>>>>> I doubt it's an oVirt bug, neither I know if it shall be
>>>>> considered a bug.
>>>>>
>>>>> We recently run a VM on a host that was memory over-used (around
>>>>> 80% of usage). The VM booted correctly, then we run "top"
and saw
>>>>> how physical RAM started decreasing every two seconds. At first
>>>>> it was 4GB, then less and less until it stabilized at around
>>>>> 600MB.
>>>>>
>>>>> Based on this (correct me if I'm wrong), we believe this is an
>>>>> effect of having this VM with ballooning enabled, as it does
>>>>> exactly this: It adds/removes RAM depending on host decision.
>>>>> Thing is that this VM had a minimal guaranteed RAM of 1GB, so
>>>>> even if this happened due to ballooning, I'm not sure if it
>>>>> should have honored the minimum guaranteed RAM.
>>>>>
>>>>> When this happened, we run a 'ps' to see with what options
the
>>>>> qemu process was invoked, and parameters seemed correct (that's
>>>>> why I don't know if it should be posted here or even if it's
a
>>>>> bug).
>>>>
>>>>
>>>>
>>>> It does belong here, it?s a different component doing it ?externally?
>>>> to
>>>> the qemu process- mom.
>>>>
>>>>>
>>>>> Is this the expected behavior?
>>>>
>>>>
>>>>
>>>> min guaranteed should be respected, adding mom maintainer Martin
>>>>
>>>> Thanks,
>>>> michal
>>>>
>>>>>
>>>>> Thanks.
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>>
>>>>
------------------------------
Message: 2
Date: Wed, 29 Mar 2017 09:46:18 +0000
From: vasily.lamykin <vasily.lamykin(a)MegaFon.ru>
To: "users(a)ovirt.org" <users(a)ovirt.org>
Subject: [ovirt-users] Memory treshold
Message-ID: <78c01f9d8162434bbebbfe6a92222ba0(a)vlg-ums05.Megafon.ru>
Content-Type: text/plain; charset="utf-8"
Hi all.
Explain please, where I can disable for cluster this threshold:
Used memory of host *** [98%] exceeded defined threshold [95%].
?
??????? ???????
??????? ??????? ?? ???????????? ????????? ????????
???????????? ????
????????? ?????? ??? ?????????
+7 (926) 500-3308
[??????? ????+???? ??? B2C]
________________________________
?????????? ? ???? ????????? ????????????? ????????????? ??? ?????????? ???, ??????? ???
??????????. ? ????????? ????? ??????????? ???????????????? ??????????, ??????? ?? ?????
???? ???????? ??? ???????????? ???-????, ????? ?????????. ???? ?? ?? ??????? ?????
?????????, ?? ?????????????, ?????????????, ??????????? ??? ??????????????? ??????????
????????? ??? ??? ????? ????????? ? ?????????. ???? ?? ???????? ??? ????????? ????????,
??????????, ??????????????? ???????? ??????????? ?? ???? ? ??????? ?? ???? ?????????? ????
????????? ? ????? ????????? ??? ????? ? ??????????.
The information contained in this communication is intended solely for the use of the
individual or entity to whom it is addressed and others authorized to receive it. It may
contain confidential or legally privileged information. The contents may not be disclosed
or used by anyone other than the addressee. If you are not the intended recipient(s), any
use, disclosure, copying, distribution or any action taken or omitted to be taken in
reliance on it is prohibited and may be unlawful. If you have received this communication
in error please notify us immediately by responding to this email and then delete the
e-mail and all attachments and any copies thereof.
(c)20mf50
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.ovirt.org/pipermail/users/attachments/20170329/5be75efd/atta...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 2130 bytes
Desc: image001.png
URL:
<http://lists.ovirt.org/pipermail/users/attachments/20170329/5be75efd/atta...
------------------------------
Message: 3
Date: Wed, 29 Mar 2017 11:59:46 +0200
From: Eduardo Mayoral <emayoral(a)arsys.es>
To: Luk?? Kaplan <lkaplan(a)dragon.cz>,users <users(a)ovirt.org>
Subject: Re: [ovirt-users] iSCSI Discovery cannot detetect LUN
Message-ID: <2e73ca85-6e54-2994-b25f-4a11a03baac4(a)arsys.es>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
I had a similar problem, in my case this was related to multipath, it was not masking the
LUNs correctly, it was seeing it multiple times (one per path), and I could not select the
LUNs in the oVirt interface.
Once I configured multipath correctly, everything worked like a charm.
Best regards,
--
Eduardo Mayoral.
On 29/03/17 11:30, Luk?? Kaplan wrote:
Hello all,
I did all steps as I described in previous email, but no change. I
can't see any LUN after discovery and login of new iSCSI storage.
(That storage is ok, if I try to connect it to another and older ovirt
domain, it is working...)
I tryed it on 3 new iSCSI targets alredy, all have same problem...
Can somebody help me, please?
--
Lukas Kaplan
2017-03-27 16:22 GMT+02:00 Luk?? Kaplan <lkaplan(a)dragon.cz
<mailto:lkaplan@dragon.cz>>:
I did following steps:
- delete target on all initiators (ovirt nodes)
iscsiadm -m node -T iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T -p
10.53.1.201:3260 <http://10.53.1.201:3260> -u
iscsiadm -m node -T iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T -p
10.53.1.201:3260 <http://10.53.1.201:3260> -o delete
- stop tgtd on target
- fill storage by zeroes (dd if=/dev/zero of=/dev/md125 bs=4096
status=progress)
- start tgtd
- tried to connect to ovirt (Discovery=ok, Login=ok, but can not
see any LUN).
=== After that I ran this commands on one node: ===
[root@fudi-cn1 ~]# iscsiadm -m session -o show
tcp: [1] 10.53.0.10:3260 <http://10.53.0.10:3260>,1
iqn.2017-03.cz.dragon.ovirt:ovirtengine (non-flash)
tcp: [11] 10.53.0.201:3260 <http://10.53.0.201:3260>,1
iqn.2017-03.cz.dragon.ovirt.fudi-sn1:10T (non-flash)
tcp: [12] 10.53.1.201:3260 <http://10.53.1.201:3260>,1
iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T (non-flash)
[root@fudi-cn1 ~]# iscsiadm -m discoverydb -P1
SENDTARGETS:
DiscoveryAddress: 10.53.0.201,3260
Target: iqn.2017-03.cz.dragon.ovirt:ovirtengine
Portal: 10.53.0.201:3260 <http://10.53.0.201:3260>,1
Iface Name: default
iSNS:
No targets found.
STATIC:
Target: iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T
Portal: 10.53.1.201:3260 <http://10.53.1.201:3260>,1
Iface Name: default
Target: iqn.2017-03.cz.dragon.ovirt:ovirtengine
Portal: 10.53.0.10:3260 <http://10.53.0.10:3260>,1
Iface Name: default
Target: iqn.2017-03.cz.dragon.ovirt.fudi-sn1:10T
Portal: 10.53.0.201:3260 <http://10.53.0.201:3260>,1
Iface Name: default
FIRMWARE:
No targets found.
=== On iscsi target: ===
[root@fuvs-sn1 ~]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md125 : active raid6 sdl1[11] sdk1[10] sdj1[9] sdi1[8] sdh1[7]
sdg1[6] sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1] sda1[0]
9766302720 blocks super 1.2 level 6, 512k chunk, algorithm 2
[12/12] [UUUUUUUUUUUU]
bitmap: 0/8 pages [0KB], 65536KB chunk
...etc...
[root@fuvs-sn1 ~]# cat /etc/tgt/targets.conf
default-driver iscsi
<target iqn.2017-03.cz.dragon.ovirt.fuvs-sn1:10T>
# provided devicce as a iSCSI target
backing-store /dev/md125
# iSCSI Initiator's IP address you allow to connect
#initiator-address 10.53.0.0/23 <http://10.53.0.0/23>
</target>
--
Lukas Kaplan
2017-03-25 12:36 GMT+01:00 Lukas Kaplan <lkaplan(a)dragon.cz
<mailto:lkaplan@dragon.cz>>:
Co muze myslet tim mappingem?
Jinak muzu zkusit ddckem celou storage prepsat nulami.
co ty na to?
Odesl?no z iPhonu
Za??tek p?epos?lan? zpr?vy:
> *Od:* Yaniv Kaul <ykaul(a)redhat.com <mailto:ykaul@redhat.com>>
> *Datum:* 24. b?ezna 2017 23:25:21 SE?
> *Komu:* Luk?? Kaplan <lkaplan(a)dragon.cz
> <mailto:lkaplan@dragon.cz>>
> *Kopie:* users <users(a)ovirt.org <mailto:users@ovirt.org>>
> *P?edm?t:* *Re: [ovirt-users] iSCSI Discovery cannot detetect
> LUN*
>
>
>
> On Fri, Mar 24, 2017 at 1:34 PM, Luk?? Kaplan
> <lkaplan(a)dragon.cz <mailto:lkaplan@dragon.cz>> wrote:
>
> Hello all,
>
> please do you have some experience with troubleshooting
> adding of iSCSI domain to ovirt 4.1.1?
>
> I am chalenging this issue now:
>
> 1) I have successfuly installed oVirt 4.1.1 environment
> with self-hosted engine, 3 nodes and 3 storages (iSCSI
> Master domain, iSCSI for hosted engine and NFS ISO
> domain). Everything is working now.
>
> 2) But, when I want to add new iSCSI domain, I can
> discover it, I can login, but I cant see any LUN on that
> storage. (I had same problem in oVirt 4.1.0, so I made
> upgrade to 4.1.1)
>
>
> Are you sure mappings are correct?
> Can you ensure the LUN is empty?
> Y.
>
>
> 3) Then I tryed to add this storage to another oVirt
> environment (oVirt 3.6) and there are no problem. I can
> see LUN on that storage and I can connect it to oVirt.
>
> I tryed to examine vdsm.log, but it is very detailed and
> unredable for me :-/
>
> Thak you in advance, have a nice day,
> --
> Lukas Kaplan
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org <mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>
>
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.ovirt.org/pipermail/users/attachments/20170329/e73ab86f/atta...
------------------------------
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
End of Users Digest, Vol 66, Issue 254
**************************************
________________________________
Информация в этом сообщении предназначена исключительно для конкретных лиц, которым она
адресована. В сообщении может содержаться конфиденциальная информация, которая не может
быть раскрыта или использована кем-либо, кроме адресатов. Если вы не адресат этого
сообщения, то использование, переадресация, копирование или распространение содержания
сообщения или его части незаконно и запрещено. Если Вы получили это сообщение ошибочно,
пожалуйста, незамедлительно сообщите отправителю об этом и удалите со всем содержимым само
сообщение и любые возможные его копии и приложения.
The information contained in this communication is intended solely for the use of the
individual or entity to whom it is addressed and others authorized to receive it. It may
contain confidential or legally privileged information. The contents may not be disclosed
or used by anyone other than the addressee. If you are not the intended recipient(s), any
use, disclosure, copying, distribution or any action taken or omitted to be taken in
reliance on it is prohibited and may be unlawful. If you have received this communication
in error please notify us immediately by responding to this email and then delete the
e-mail and all attachments and any copies thereof.
(c)20mf50