Hi Marko,

 I have also had this problem on an oVirt 4.3 install - I could never fix it and have not tried to upgrade to 4.4 yet - my workaround for now (as you have deduced) is to just 'up' it while running so its hotswapped in - annoying, but I think the best way around this without upgrade would be to re-deploy the hosted engine defining the new resources using an engine-backup - but again this is just as scary as in-place upgrade lest you lock yourself out from your other running VMs in a production environment.

 I plan to do it - I just need some time to set it up in a lab so I know what to expect.

Thanks,
Joe

On 23/02/2021 5:05 pm, Vrgotic, Marko wrote:

From the logs, the XML file is updated with new memory value:

2021-02-23 05:41:53,350Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.DumpXmlsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] FINISH, DumpXmlsVDSCommand, return: {e3de2b68-b7ea-4466-b044-ed431267df1b=<domain type='kvm' id='24'>

  <name>HostedEngine</name>

  <uuid>e3de2b68-b7ea-4466-b044-ed431267df1b</uuid>

  <metadata xmlns:ns0="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0">

    <ns0:qos/>

    <ovirt-vm:vm xmlns:ovirt-vm="http://ovirt.org/vm/1.0">

    <ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion>

    <ovirt-vm:destroy_on_reboot type="bool">False</ovirt-vm:destroy_on_reboot>

    <ovirt-vm:launchPaused>false</ovirt-vm:launchPaused>

    <ovirt-vm:memGuaranteedSize type="int">16384</ovirt-vm:memGuaranteedSize>

    <ovirt-vm:minGuaranteedMemoryMb type="int">1024</ovirt-vm:minGuaranteedMemoryMb>

    <ovirt-vm:resumeBehavior>auto_resume</ovirt-vm:resumeBehavior>

    <ovirt-vm:startTime type="float">1614004416.2</ovirt-vm:startTime>

    <ovirt-vm:device mac_address="00:16:3e:4f:2a:6d">

        <ovirt-vm:network>ovirtmgmt</ovirt-vm:network>

        <ovirt-vm:custom>

            <ovirt-vm:queues>4</ovirt-vm:queues>

        </ovirt-vm:custom>

    </ovirt-vm:device>

    <ovirt-vm:device devtype="disk" name="vda">

        <ovirt-vm:domainID>84dbd790-39af-440c-acea-b0db03bf86b0</ovirt-vm:domainID>

        <ovirt-vm:guestName>/dev/vda2</ovirt-vm:guestName>

        <ovirt-vm:imageID>42b45aba-2307-4ef7-9465-533f95946b0a</ovirt-vm:imageID>

        <ovirt-vm:poolID>00000000-0000-0000-0000-000000000000</ovirt-vm:poolID>

        <ovirt-vm:shared>exclusive</ovirt-vm:shared>

        <ovirt-vm:volumeID>38163651-2197-4661-9c9d-84c4272d6775</ovirt-vm:volumeID>

        <ovirt-vm:specParams>

            <ovirt-vm:pinToIoThread>1</ovirt-vm:pinToIoThread>

        </ovirt-vm:specParams>

        <ovirt-vm:volumeChain>

            <ovirt-vm:volumeChainNode>

                <ovirt-vm:domainID>84dbd790-39af-440c-acea-b0db03bf86b0</ovirt-vm:domainID>

                <ovirt-vm:imageID>42b45aba-2307-4ef7-9465-533f95946b0a</ovirt-vm:imageID>

                <ovirt-vm:leaseOffset type="int">0</ovirt-vm:leaseOffset>

                <ovirt-vm:leasePath>/rhev/data-center/mnt/172.17.28.5:_ovirt__hosted__engine/84dbd790-39af-440c-acea-b0db03bf86b0/images/42b45aba-2307-4ef7-9465-533f95946b0a/38163651-2197-4661-9c9d-84c4272d6775.lease</ovirt-vm:leasePath>

                <ovirt-vm:path>/rhev/data-center/mnt/172.17.28.5:_ovirt__hosted__engine/84dbd790-39af-440c-acea-b0db03bf86b0/images/42b45aba-2307-4ef7-9465-533f95946b0a/38163651-2197-4661-9c9d-84c4272d6775</ovirt-vm:path>

                <ovirt-vm:volumeID>38163651-2197-4661-9c9d-84c4272d6775</ovirt-vm:volumeID>

            </ovirt-vm:volumeChainNode>

        </ovirt-vm:volumeChain>

    </ovirt-vm:device>

    <ovirt-vm:device devtype="disk" name="hdc"/>

</ovirt-vm:vm>

  </metadata>

  <maxMemory slots='16' unit='KiB'>67108864</maxMemory>

  <memory unit='KiB'>33554432</memory>

  <currentMemory unit='KiB'>33554432</currentMemory>

  <vcpu placement='static' current='8'>64</vcpu>

  <vcpus>

    <vcpu id='0' enabled='yes' hotpluggable='no' order='1'/>

    <vcpu id='1' enabled='yes' hotpluggable='no' order='2'/>

    <vcpu id='2' enabled='yes' hotpluggable='no' order='3'/>

    <vcpu id='3' enabled='yes' hotpluggable='no' order='4'/>

    <vcpu id='4' enabled='yes' hotpluggable='yes' order='5'/>

    <vcpu id='5' enabled='yes' hotpluggable='yes' order='6'/>

    <vcpu id='6' enabled='yes' hotpluggable='yes' order='7'/>

    <vcpu id='7' enabled='yes' hotpluggable='yes' order='8'/>

    <vcpu id='8' enabled='no' hotpluggable='yes'/>

    <vcpu id='9' enabled='no' hotpluggable='yes'/>

 

But in UI, I still see

 

Defined Memory:

16384 MB

 

 

And the vm.conf on all three Hosts is still showing

smp=8                       <= cpu increase is already updated

memSize=16384     <= but ram is still same

 

 

It looks like  should report a bug. But before I do that, we are currently on 4.3.8.2 version – is this by any change fixed by latest 4.3 version?

 

 

-----

kind regards/met vriendelijke groeten

 

Marko Vrgotic
Sr. System Engineer @ System Administration


ActiveVideo

o: +31 (35) 6774131

m: +31 (65) 5734174

e: m.vrgotic@activevideo.com
w: www.activevideo.com

 

ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

 

 

 

From: Vrgotic, Marko <M.Vrgotic@activevideo.com>
Date: Monday, 22 February 2021 at 21:30
To: Yedidyah Bar David <didi@redhat.com>
Cc: users@ovirt.org <users@ovirt.org>
Subject: Re: [ovirt-users] oVirt Engine increase RAM and CPU

Mem guaranteed updates,

CPU cores updates instantly and perist after reboot

But defined memory is “stuck” on 16384 after reboot.

 

 

 

 

 

-----

kind regards/met vriendelijke groeten

 

Marko Vrgotic
Sr. System Engineer @ System Administration


ActiveVideo

o: +31 (35) 6774131

m: +31 (65) 5734174

e: m.vrgotic@activevideo.com
w: www.activevideo.com

 

ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

 

 

 

From: Vrgotic, Marko <M.Vrgotic@activevideo.com>
Date: Monday, 22 February 2021 at 20:57
To: Yedidyah Bar David <didi@redhat.com>
Cc: users@ovirt.org <users@ovirt.org>
Subject: Re: [ovirt-users] oVirt Engine increase RAM and CPU

Hi all,

 

Still testing in staging and I have not moved from updating RAM.

Initially, I made the change via Portal, updating defined memory of HE from 16384 to 32768. The change is instantly visible on VM when I execute free -m , but in the VM info it still says 16384.

I waited almost 4hours and this value did not get updated. Tried shutdown, remove GM, allow the HA to start the VM, it booted with 16384 allocated.

 

Than I did it again, and tried with Update OVF from storage domain where the HE disk is, same, no change.

 

Tried, updating memory again, migrating to another HA host – Update OVF , same thing, after reboot, the memory is back to 16384.

 

Can somebody give me an advice where to look further? Again, making the change in memory amount is instant on the VM, but after reboot, its back to 16384.

 

 

-----

kind regards/met vriendelijke groeten

 

Marko Vrgotic
Sr. System Engineer @ System Administration


ActiveVideo

o: +31 (35) 6774131

m: +31 (65) 5734174

e: m.vrgotic@activevideo.com
w: www.activevideo.com

 

ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

 

 

 

From: Yedidyah Bar David <didi@redhat.com>
Date: Monday, 22 February 2021 at 15:19
To: Vrgotic, Marko <M.Vrgotic@activevideo.com>
Cc: users@ovirt.org <users@ovirt.org>
Subject: Re: [ovirt-users] oVirt Engine increase RAM and CPU

***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!***

On Mon, Feb 22, 2021 at 4:10 PM Vrgotic, Marko
<M.Vrgotic@activevideo.com> wrote:
>
> Hi Didi,
>
>
>
> I increased the RAM from 16 to 32 and change is instant on the VM, but as you have mentioned, it does not get written right away to VM image.
>
> Upon reboot its back to 16GB RAM.
>
>
>
>
>
> At this moment, change is applied, and I am waiting up to 60min to see if the changes will be written to survive reboot.
>
>
>
> Regarding how to change it, the UI Edit seem to be the way. If I go with /var/run/… file changes fast, even with VM being shutdown.
>
>
>
> @oVirt does anyone know how long it usually takes to have the change written to VM image?

up to 60 minutes :-).

I think you can change this with engine-config, item
OvfUpdateIntervalInMinutes .

Best regards,

>
>
>
>
>
>
>
> -----
>
> kind regards/met vriendelijke groeten
>
>
>
> Marko Vrgotic
> Sr. System Engineer @ System Administration
>
>
> ActiveVideo
>
> o: +31 (35) 6774131
>
> m: +31 (65) 5734174
>
> e: m.vrgotic@activevideo.com
> w: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.activevideo.com%2F&amp;data=04%7C01%7Cm.vrgotic%40activevideo.com%7C7973b1e3b1f6400c1bd608d8d73cd3d2%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637496003548256181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0TuDZFRZuYvOwEbsSflrSiXuKJ2dt2UIcys6UzDzSsA%3D&amp;reserved=0
>
>
>
> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
>
>
>
>
>
>
>
> From: Yedidyah Bar David <didi@redhat.com>
> Date: Monday, 22 February 2021 at 11:05
> To: Vrgotic, Marko <M.Vrgotic@activevideo.com>
> Cc: users@ovirt.org <users@ovirt.org>
> Subject: Re: [ovirt-users] oVirt Engine increase RAM and CPU
>
> ***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!***
>
> On Mon, Feb 22, 2021 at 11:01 AM Vrgotic, Marko
> <M.Vrgotic@activevideo.com> wrote:
> >
> > Hi Didi,
> >
> >
> >
> > Just to be sure,  considering its an HA instance, I need to set to maintenance=global and execute these changes – CPU increase will most likely require reboot?
>
> Please note that I do not have any experience with this whatsoever.
> Also for the further replies below. Just guessing. I suggest to try
> first on a test system if it's about a production/critical system.
>
> I guess that it will mark the VM as requiring a reboot for changes to
> take effect, and that they'll actually be applied only on hard-reboot
> (meaning, not just running 'reboot' from inside). So yes,
> I guess the process should be:
>
> 1. Edit as needed
> 2. Perhaps wait a bit or something like that to make sure changes
> reached OVF_STORE on the shared storage. No idea how. Perhaps we log
> enough about this to existing logs, or perhaps you can dump the conf
> to verify manually.
> 2. Set global maint, shutdown, clear global maint (and either
> immediately start the vm or let ha-agent do this for you).
>
> >
> >
> >
> > For the HEAP changes, I have been doing that lot on the Atlassian instances, just not sure how oVirt-engine  is going to react and if there are any guidelines I should stick to in ovirt  case.
>
> I have no idea other than the bugs I was personally involved with
> (which you can find in bugzilla).
> Perhaps RHV or oVirt docs have some guidelines, didn't check.
> If not, then I suppose the guidelines are "Use the defaults and follow
> recommendations", which mean:
> 1. Allocate 16GB RAM for the HE VM. Perhaps we even recommend more
> than that for larger setups, not sure.
> 2. engine-setup then sets both min and max to 4GB (=25% of RAM).
>
> If you do want to play with this, you should of course test thoroughly
> and carefully :-).
> Consider also other processes on the machine needing rather-large
> amounts of memory, including DWH and PostgreSQL (which we also
> automatically configure and which you can further refine if needed,
> and also set up to run on separate machines).
>
> >
> > I.e. What if I set min and max to same value?
>
> That's a question for the JVM, not for oVirt :-). I suppose it will
> simply use a constant size.
>
> >
> > Is it going to be “ok” if I assign more than half of system RAM to the JAVA? Ect…
>
> It should be, if you make sure other stuff (dwh+pg) do not use too
> much, or you have also enough swap (but then do not expect good
> performance if you actually run into actually using it).
>
> >
> >
> >
> > -----
> >
> > kind regards/met vriendelijke groeten
> >
> >
> >
> > Marko Vrgotic
> > Sr. System Engineer @ System Administration
> >
> >
> > ActiveVideo
> >
> > o: +31 (35) 6774131
> >
> > m: +31 (65) 5734174
> >
> > e: m.vrgotic@activevideo.com
> > w: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.activevideo.com%2F&amp;data=04%7C01%7Cm.vrgotic%40activevideo.com%7C7973b1e3b1f6400c1bd608d8d73cd3d2%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637496003548256181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=0TuDZFRZuYvOwEbsSflrSiXuKJ2dt2UIcys6UzDzSsA%3D&amp;reserved=0
> >
> >
> >
> > ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
> >
> >
> >
> >
> >
> >
> >
> > From: Vrgotic, Marko <M.Vrgotic@activevideo.com>
> > Date: Monday, 22 February 2021 at 09:08
> > To: Yedidyah Bar David <didi@redhat.com>
> > Cc: users@ovirt.org <users@ovirt.org>
> > Subject: Re: [ovirt-users] oVirt Engine increase RAM and CPU
> >
> > Thank you Didi.
> >
> >
> >
> > -----
> >
> > kind regards/met vriendelijke groeten
> >
> >
> >
> > Marko Vrgotic
> > Sr. System Engineer @ System Administration
> >
> >
> > ActiveVideo
> >
> > o: +31 (35) 6774131
> >
> > m: +31 (65) 5734174
> >
> > e: m.vrgotic@activevideo.com
> > w: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.activevideo.com%2F&amp;data=04%7C01%7Cm.vrgotic%40activevideo.com%7C7973b1e3b1f6400c1bd608d8d73cd3d2%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637496003548266178%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=7CdjxCvgBO4kAHoTZGxEOWar8KkboetvlPhcnnhG2Mc%3D&amp;reserved=0
> >
> >
> >
> > ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
> >
> >
> >
> >
> >
> >
> >
> > From: Yedidyah Bar David <didi@redhat.com>
> > Date: Sunday, 21 February 2021 at 08:03
> > To: Vrgotic, Marko <M.Vrgotic@activevideo.com>
> > Cc: users@ovirt.org <users@ovirt.org>
> > Subject: Re: [ovirt-users] oVirt Engine increase RAM and CPU
> >
> > ***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!***
> >
> > On Fri, Feb 19, 2021 at 12:39 PM Vrgotic, Marko
> > <M.Vrgotic@activevideo.com> wrote:
> > >
> > > Hi oVirt,
> > >
> > >
> > >
> > > What is the cleanest way to increase the memory and cpu for SHE engine?
> >
> > Did you try to simply edit the vm from the admin ui? This should be
> > possible for most fields for quite some time now:
> >
> > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1160094&amp;data=04%7C01%7Cm.vrgotic%40activevideo.com%7C7973b1e3b1f6400c1bd608d8d73cd3d2%7C214268a3e1214486acd4545c9faf2252%7C0%7C0%7C637496003548266178%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=uoeUaAZMwneu0Pq1pXKYKFkKpMAsF0znhcgTIxZpbso%3D&amp;reserved=0
> >
> > >
> > >
> > >
> > > We have the Engine running on top of three HA hosts, so I guess the specs need to be change on all three, while engine is down.
> > >
> > >
> > >
> > > Is it done by editing the VM libvirt conf file or some other way?
> > >
> > >
> > >
> > > Where do I allocate more RAM to JAVA for oVirt?
> >
> > You can control the values passed as -Xms and -Xmx using the config
> > values ENGINE_HEAP_MIN and ENGINE_HEAP_MAX.
> >
> > They are set by engine-setup on initial setup but not changed later.
> > It writes /etc/ovirt-engine/engine.conf.d/10-setup-java.conf .
> > It's probably safer to create your own file for overriding, e.g.
> > 99-my-setup-java.conf or something.
> >
> > There have been various relevant changes to this over the years, you
> > can search bugzilla for 'ENGINE_HEAP_MIN' to find them.
> >
> > >
> > >
> > >
> > > We are currently running 4.3.8 version.
> > >
> > >
> > >
> > > Kindly awaiting your reply.
> >
> > Good luck and best regards,
> > --
> > Didi
>
>
>
> --
> Didi



--
Didi


_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/W7MM4YPD7JX3QRW32M7OFG7EIGCGALVY/