[ovirt-users] engine tries to allocate 11GB and aborts (was: hosted engine does not start correctly (CentOS 7.1, overt-engine 3.5.2))
H. Haven Liu
haven.liu at ucla.edu
Tue May 26 19:26:07 UTC 2015
Both source and destination machines (assuming you are referring to the
physical host source and virtual host destination of the migration) are
running ovirt-engine 22.214.171.124-1. However, the "source" is on CentOS 6.6,
whereas the "destination" is on CentOS 7.1. The source is running
postgresql(/-server) 8.4.20, Java 1.7.0_79, httpd 2.2.15. Destination is
running postgresql(/-server) 9.2.10, Java 126.96.36.199, jboss-httpd 2.4.6. I'm
not sure which over package version is relevant.
The failure occurs after engine-setup, and the web UI doesn't load, and the
host reports the engine is not up (I'm not in front of the host, so I don't
have the exact language).
Our current setup has 2 physical hosts, ~15 active VMs. Each host has 64GB
RAM. Even with all the VM on one host, the utilization is less than 50%. We
have another physical storage server, that provides the NFS storage backend
to the hosts and VMs. Prior to attempting the hosted-engine migration, I
migrated all the VMs to host #1, where the engine was originally installed.
I then removed host #2 from the host list via the UI, and performed
hosted-engine --deploy on host #2. A few other quirks I encountered:
1. I originally wanted to do "hosted-engine --deploy" on host #1, but that
it was unsuccessful due to it complaining the storage connection was busy
2. I then tried to do "hosted-engine --deploy" on host #2, using the same
storage connection path as what I used for host #1 originally. The setup
process asked if it was an "additional host", I said no, but later it
cannot activate the storage connection.
3. Lastly I did "hosted-engine --deploy" on host #2, but pointing to a
different storage connection path, still on the same storage server. This
was successful until the I encountered the failure in question.
On Tue, May 26, 2015 at 2:20 AM, Yedidyah Bar David <didi at redhat.com> wrote:
> ----- Original Message -----
> > From: "Haoyang Haven Liu" <haven.liu at ucla.edu>
> > To: users at ovirt.org
> > Sent: Tuesday, May 26, 2015 8:34:00 AM
> > Subject: [ovirt-users] hosted engine does not start correctly (CentOS
> 7.1, overt-engine 3.5.2)
> > Hello,
> > I tried to follow < http://www.ovirt.org/Migrate_to_Hosted_Engine > to
> > migrate an existing ovirt environment. I managed to complete the
> > “engine-backup —mode=restore…” step, and the following “engine-setup”
> > appear to have completed. However, the ovirt-engine service was not
> > successfully has errors:
> > [root at ovirt ~]# systemctl status ovirt-engine
> > ovirt-engine.service - oVirt Engine
> > Loaded: loaded (/usr/lib/systemd/system/ovirt-engine.service; enabled)
> > Active: failed (Result: exit-code) since Mon 2015-05-25 22:26:12 PDT;
> 43s ago
> > Main PID: 15931 (code=exited, status=1/FAILURE)
> > May 25 22:26:11 ovirt.polysep.ucla.edu systemd: Started oVirt Engine.
> > May 25 22:26:12 ovirt.polysep.ucla.edu ovirt-engine.py:
> > 22:26:12,269 ovirt-engine: ERROR run:532 Error: process termina...code 1
> > May 25 22:26:12 ovirt.polysep.ucla.edu systemd:
> ovirt-engine.service: main
> > process exited, code=exited, status=1/FAILURE
> > May 25 22:26:12 ovirt.polysep.ucla.edu systemd: Unit
> > entered failed state.
> > Hint: Some lines were ellipsized, use -l to show in full.
> > Additionally, /var/log/ovirt-engine/console.log complains “cannot
> > memory”. The VM engine has 4GB of memory, but ovirt-engine engine seems
> > want ~11GB of memory, is that right?
> > [root at ovirt ~]# cat /var/log/ovirt-engine/console.log
> > OpenJDK 64-Bit Server VM warning: INFO:
> > 11250696192, 0) failed; error='Cannot allocate memory' (errno=12)
> > #
> > # There is insufficient memory for the Java Runtime Environment to
> > # Native memory allocation (malloc) failed to allocate 11250696192 bytes
> > committing reserved memory.
> > # An error report file with more information is saved as:
> > # /tmp/jvm-15974/hs_error.log
> > [root at ovirt ~]# free -m
> > total used free shared buff/cache available
> > Mem: 3791 123 3175 28 492 3428
> > Swap: 2559 0 2559
> > [root at ovirt ~]#
> > The hs_error.log is attached.
> > Any help is appreciated.
> Can you please compare versions of all installed packages between the
> and destination machines?
> I assume the failure above is right after engine-setup, right? Before
> the interaction on hosted-engine side?
> Did you try to restart the engine (systemctl restart ovirt-engine)? Did it
> the same way?
> How much memory do you have on the source machine? And how much does the
> actually use there?
> How large is your system (hosts, VMs etc)?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Users