4:14 p.m.
On Mon, Dec 7, 2015 at 4:00 PM, Melissa Mesler <melissa(a)aasteel.com> wrote:
Here is what I retrieved from the postresql logs:
LOG: database system was interrupted; last known up at 2015-12-04 10:57:43
CST
LOG: unexpected pageaddr 0/69564000 in log file 0, segment 107, offset
5652480
LOG: invalid primary checkpoint record
LOG: unexpected pageaddr 0/6952A000 in log file 0, segment 107, offset
5414912
LOG: invalid secondary checkpoint record
PANIC: could not locate a valid checkpoint record
LOG: startup process (PID 1637) was terminated by signal 6: Aborted
LOG: aborting startup due to startup process failure
LOG: database system was interrupted; last known up at 2015-12-04 10:57:43
CST
LOG: unexpected pageaddr 0/69564000 in log file 0, segment 107, offset
5652480
LOG: invalid primary checkpoint record
LOG: unexpected pageaddr 0/6952A000 in log file 0, segment 107, offset
5414912
LOG: invalid secondary checkpoint record
PANIC: could not locate a valid checkpoint record
LOG: startup process (PID 2632) was terminated by signal 6: Aborted
LOG: aborting startup due to startup process failure
Also, when I did the sync on the ovirt vm it was powered off. I know not to
do live syncs as there could be issues. That's why I'm so confused that
there's a pr oblem.
No idea, adding Eli and changing subject.
Best,
On 12/7/15 1:03 AM, Yedidyah Bar David wrote:
On Mon, Dec 7, 2015 at 12:09 AM, Melissa Mesler <melissa(a)aasteel.com> wrote:
Hopefully someone can give me a good idea of how to troubleshoot an ovirt
issue. I am running 3.5 on CentOS 6.7. Here is the issue I'm having. I have
Ovirt running as a virtual machine in KVM. I recently moved (with virtsync)
the vm to rebuild the host. Once the host was rebuilt I moved ovirt back.
Now ovirt-engine starts but is inaccessible. In the logs it says this:
2015-12-06 15:57:24,558 ERROR [org.ovirt.engine.core.bll.Backend] (MSC
service thread 1-2) Error in getting DB connection. The database is
inaccessible. Original exception is: DataAccessResourceFailureException:
Error retreiving database metadata; nested exception is
org.springframework.jdbc.support.MetaDataAccessException: Could not get
Connection for extracting meta data; nested exception is
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
JDBC Connection; nested exception is java.sql.SQLException:
javax.resource.ResourceException: IJ000453: Unable to get managed connection
for java:/ENGINEDataSource
I did notice that I can't get postgresql to start.
What happens when you try? Check/post also pg logs please, from
/var/lib/pgsql/data/pg_log/ .
Everything worked
perfectly fine before. Any ideas?
Was the copy done from a live machine? From a snapshot? Perhaps it wasn't
consistent? Need to check pg logs.
Best,
--
Didi
5:58 p.m.
----- Original Message -----
From: "Yedidyah Bar David" <didi(a)redhat.com>
To: "Melissa Mesler" <melissa(a)aasteel.com>, "Eli Mesika"
<emesika(a)redhat.com>
Cc: users(a)ovirt.org
Sent: Monday, December 7, 2015 4:14:49 PM
Subject: Postgresql failure (was: [ovirt-users] Ovirt Engine Issue)
On Mon, Dec 7, 2015 at 4:00 PM, Melissa Mesler <melissa(a)aasteel.com> wrote:
> Here is what I retrieved from the postresql logs:
>
> LOG: database system was interrupted; last known up at 2015-12-04 10:57:43
> CST
> LOG: unexpected pageaddr 0/69564000 in log file 0, segment 107, offset
> 5652480
> LOG: invalid primary checkpoint record
> LOG: unexpected pageaddr 0/6952A000 in log file 0, segment 107, offset
> 5414912
> LOG: invalid secondary checkpoint record
> PANIC: could not locate a valid checkpoint record
> LOG: startup process (PID 1637) was terminated by signal 6: Aborted
> LOG: aborting startup due to startup process failure
> LOG: database system was interrupted; last known up at 2015-12-04 10:57:43
> CST
> LOG: unexpected pageaddr 0/69564000 in log file 0, segment 107, offset
> 5652480
> LOG: invalid primary checkpoint record
> LOG: unexpected pageaddr 0/6952A000 in log file 0, segment 107, offset
> 5414912
> LOG: invalid secondary checkpoint record
> PANIC: could not locate a valid checkpoint record
> LOG: startup process (PID 2632) was terminated by signal 6: Aborted
> LOG: aborting startup due to startup process failure
>
> Also, when I did the sync on the ovirt vm it was powered off. I know not to
> do live syncs as there could be issues. That's why I'm so confused that
> there's a pr oblem.
No idea, adding Eli and changing subject.
Seems as database is corrupted
This is a serious issue that someone with DBA knowledge should handle
http://blog.endpoint.com/2014/11/when-postgres-will-not-start.html
Remember to do a physical copy of your database file to a different partition or server
before you start
Eli
Best,
>
>
>
> On 12/7/15 1:03 AM, Yedidyah Bar David wrote:
>
> On Mon, Dec 7, 2015 at 12:09 AM, Melissa Mesler <melissa(a)aasteel.com>
> wrote:
>
> Hopefully someone can give me a good idea of how to troubleshoot an ovirt
> issue. I am running 3.5 on CentOS 6.7. Here is the issue I'm having. I have
> Ovirt running as a virtual machine in KVM. I recently moved (with virtsync)
> the vm to rebuild the host. Once the host was rebuilt I moved ovirt back.
> Now ovirt-engine starts but is inaccessible. In the logs it says this:
>
> 2015-12-06 15:57:24,558 ERROR [org.ovirt.engine.core.bll.Backend] (MSC
> service thread 1-2) Error in getting DB connection. The database is
> inaccessible. Original exception is: DataAccessResourceFailureException:
> Error retreiving database metadata; nested exception is
> org.springframework.jdbc.support.MetaDataAccessException: Could not get
> Connection for extracting meta data; nested exception is
> org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
> JDBC Connection; nested exception is java.sql.SQLException:
> javax.resource.ResourceException: IJ000453: Unable to get managed
> connection
> for java:/ENGINEDataSource
>
> I did notice that I can't get postgresql to start.
>
> What happens when you try? Check/post also pg logs please, from
> /var/lib/pgsql/data/pg_log/ .
>
> Everything worked
> perfectly fine before. Any ideas?
>
> Was the copy done from a live machine? From a snapshot? Perhaps it wasn't
> consistent? Need to check pg logs.
>
> Best,
>
>
--
Didi
6:51 p.m.
New subject: Postgresql failure
This is a multi-part message in MIME format.
--------------090202020108020901020009
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
I might end up just rebuilding my ovirt which isn't a huge deal as my
set up is pretty simple.
I do have a question as to why this would happen. Can anyone give me
guidance there? I would understand if I forcefully shut down the system
but I didn't. I always shut my systems down clean so the database
portion should've shutdown that way also. Just trying to understand more
of why possibly this would happen so that I could avoid this in the future.
---
Melissa Mesler
On 12/7/15 8:14 AM, Yedidyah Bar David wrote:
On Mon, Dec 7, 2015 at 4:00 PM, Melissa Mesler
<melissa(a)aasteel.com> wrote:
> Here is what I retrieved from the postresql logs:
>
> LOG: database system was interrupted; last known up at 2015-12-04 10:57:43
> CST
> LOG: unexpected pageaddr 0/69564000 in log file 0, segment 107, offset
> 5652480
> LOG: invalid primary checkpoint record
> LOG: unexpected pageaddr 0/6952A000 in log file 0, segment 107, offset
> 5414912
> LOG: invalid secondary checkpoint record
> PANIC: could not locate a valid checkpoint record
> LOG: startup process (PID 1637) was terminated by signal 6: Aborted
> LOG: aborting startup due to startup process failure
> LOG: database system was interrupted; last known up at 2015-12-04 10:57:43
> CST
> LOG: unexpected pageaddr 0/69564000 in log file 0, segment 107, offset
> 5652480
> LOG: invalid primary checkpoint record
> LOG: unexpected pageaddr 0/6952A000 in log file 0, segment 107, offset
> 5414912
> LOG: invalid secondary checkpoint record
> PANIC: could not locate a valid checkpoint record
> LOG: startup process (PID 2632) was terminated by signal 6: Aborted
> LOG: aborting startup due to startup process failure
>
> Also, when I did the sync on the ovirt vm it was powered off. I know not to
> do live syncs as there could be issues. That's why I'm so confused that
> there's a pr oblem.
No idea, adding Eli and changing subject.
Best,
>
>
> On 12/7/15 1:03 AM, Yedidyah Bar David wrote:
>
> On Mon, Dec 7, 2015 at 12:09 AM, Melissa Mesler <melissa(a)aasteel.com> wrote:
>
> Hopefully someone can give me a good idea of how to troubleshoot an ovirt
> issue. I am running 3.5 on CentOS 6.7. Here is the issue I'm having. I have
> Ovirt running as a virtual machine in KVM. I recently moved (with virtsync)
> the vm to rebuild the host. Once the host was rebuilt I moved ovirt back.
> Now ovirt-engine starts but is inaccessible. In the logs it says this:
>
> 2015-12-06 15:57:24,558 ERROR [org.ovirt.engine.core.bll.Backend] (MSC
> service thread 1-2) Error in getting DB connection. The database is
> inaccessible. Original exception is: DataAccessResourceFailureException:
> Error retreiving database metadata; nested exception is
> org.springframework.jdbc.support.MetaDataAccessException: Could not get
> Connection for extracting meta data; nested exception is
> org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
> JDBC Connection; nested exception is java.sql.SQLException:
> javax.resource.ResourceException: IJ000453: Unable to get managed connection
> for java:/ENGINEDataSource
>
> I did notice that I can't get postgresql to start.
>
> What happens when you try? Check/post also pg logs please, from
> /var/lib/pgsql/data/pg_log/ .
>
> Everything worked
> perfectly fine before. Any ideas?
>
> Was the copy done from a live machine? From a snapshot? Perhaps it wasn't
> consistent? Need to check pg logs.
>
> Best,
>
>
--------------090202020108020901020009
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<small>I might end up just rebuilding my ovirt which isn't a huge
deal as my set up is pretty simple.<br>
<br>
I do have a question as to why this would happen. Can anyone give
me guidance there? I would understand if I forcefully shut down
the system but I didn't. I always shut my systems down clean so
the database portion should've shutdown that way also. Just trying
to understand more of why possibly this would happen so that I
could avoid this in the future.<br>
<br>
</small>
<pre class="moz-signature" cols="72">---
Melissa Mesler
</pre>
<div class="moz-cite-prefix">On 12/7/15 8:14 AM, Yedidyah Bar David
wrote:<br>
</div>
<blockquote
cite="mid:CAHRwYXtrUK1XVMRKTnHGA40ML0w--1XxArm4CKBZAfmvq2X0zQ@mail.gmail.com"
type="cite">
<pre wrap="">On Mon, Dec 7, 2015 at 4:00 PM, Melissa Mesler <a
class="moz-txt-link-rfc2396E"
href="mailto:melissa@aasteel.com"><melissa@aasteel.com></a>
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Here is what I retrieved from the postresql logs:
LOG: database system was interrupted; last known up at 2015-12-04 10:57:43
CST
LOG: unexpected pageaddr 0/69564000 in log file 0, segment 107, offset
5652480
LOG: invalid primary checkpoint record
LOG: unexpected pageaddr 0/6952A000 in log file 0, segment 107, offset
5414912
LOG: invalid secondary checkpoint record
PANIC: could not locate a valid checkpoint record
LOG: startup process (PID 1637) was terminated by signal 6: Aborted
LOG: aborting startup due to startup process failure
LOG: database system was interrupted; last known up at 2015-12-04 10:57:43
CST
LOG: unexpected pageaddr 0/69564000 in log file 0, segment 107, offset
5652480
LOG: invalid primary checkpoint record
LOG: unexpected pageaddr 0/6952A000 in log file 0, segment 107, offset
5414912
LOG: invalid secondary checkpoint record
PANIC: could not locate a valid checkpoint record
LOG: startup process (PID 2632) was terminated by signal 6: Aborted
LOG: aborting startup due to startup process failure
Also, when I did the sync on the ovirt vm it was powered off. I know not to
do live syncs as there could be issues. That's why I'm so confused that
there's a pr oblem.
</pre>
</blockquote>
<pre wrap="">
No idea, adding Eli and changing subject.
Best,
</pre>
<blockquote type="cite">
<pre wrap="">
On 12/7/15 1:03 AM, Yedidyah Bar David wrote:
On Mon, Dec 7, 2015 at 12:09 AM, Melissa Mesler <a
class="moz-txt-link-rfc2396E"
href="mailto:melissa@aasteel.com"><melissa@aasteel.com></a>
wrote:
Hopefully someone can give me a good idea of how to troubleshoot an ovirt
issue. I am running 3.5 on CentOS 6.7. Here is the issue I'm having. I have
Ovirt running as a virtual machine in KVM. I recently moved (with virtsync)
the vm to rebuild the host. Once the host was rebuilt I moved ovirt back.
Now ovirt-engine starts but is inaccessible. In the logs it says this:
2015-12-06 15:57:24,558 ERROR [org.ovirt.engine.core.bll.Backend] (MSC
service thread 1-2) Error in getting DB connection. The database is
inaccessible. Original exception is: DataAccessResourceFailureException:
Error retreiving database metadata; nested exception is
org.springframework.jdbc.support.MetaDataAccessException: Could not get
Connection for extracting meta data; nested exception is
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
JDBC Connection; nested exception is java.sql.SQLException:
javax.resource.ResourceException: IJ000453: Unable to get managed connection
for java:/ENGINEDataSource
I did notice that I can't get postgresql to start.
What happens when you try? Check/post also pg logs please, from
/var/lib/pgsql/data/pg_log/ .
Everything worked
perfectly fine before. Any ideas?
Was the copy done from a live machine? From a snapshot? Perhaps it wasn't
consistent? Need to check pg logs.
Best,
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<br>
</body>
</html>
--------------090202020108020901020009--
3285
days inactive
3285
days old
2 comments
3 participants
participants (3)
-
Eli Mesika -
Melissa Mesler -
Yedidyah Bar David