From: "Neil" <nwilson123(a)gmail.com&gt; To: users(a)ovirt.org Sent: Monday, October 15, 2012 9:27:24 AM Subject: [Users] Fwd: database issue after power failure I've found out now that another technician accidentally deleted my psql engine db because he tried to install bacula on the same server. I have no backup of the DB because I was still in the process of setting them up and wasn't aware he was going to be installing onto this server. I currently have 6 guests that are in a shutdown state because of a power failure over the weekend. It's using FC storage, only the db seems to be missing. Is there anything I can do to recover from this disaster?

Thanks. Regards. Neil. ---------- Forwarded message ---------- From: Neil <nwilson123(a)gmail.com&gt; Date: Mon, Oct 15, 2012 at 9:10 AM Subject: database issue after power failure To: users(a)ovirt.org Hi guys, I've got what looks like a huge issue after a power failure. In my server.log I'm seeing "Caused by: org.postgresql.util.PSQLException: FATAL: database "engine" does not exist" in my server.log and in my engine.log I'm seeing the following which also looks very bad. "2012-10-15 09:08:41,357 ERROR [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Error in getting DB connection. The database is inaccessible. Original exception is: DataAccessResourceFailureException: Error retreiving database metadata; nested exception is org.springframework.jdbc.support.MetaDataAccessException: Could not get Connection for extracting meta data; nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource" Please could someone assist if possible, I'm currently logged in on #ovirt as well. Thanks. Regards. Neil Wilson. _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

On Mon, Oct 15, 2012 at 11:43 AM, Eli Mesika <emesika(a)redhat.com&gt; wrote: > > > ----- Original Message ----- >> From: "Neil" <nwilson123(a)gmail.com&gt; >> To: users(a)ovirt.org >> Sent: Monday, October 15, 2012 9:27:24 AM >> Subject: [Users] Fwd: database issue after power failure >> >> I've found out now that another technician accidentally deleted my >> psql engine db because he tried to install bacula on the same server. >> I have no backup of the DB because I was still in the process of >> setting them up and wasn't aware he was going to be installing onto >> this server. >> >> I currently have 6 guests that are in a shutdown state because of a >> power failure over the weekend. It's using FC storage, only the db >> seems to be missing. >> >> Is there anything I can do to recover from this disaster? > > Hi > Is this a new installation or one that was upgraded? > If it is an upgrade IIRC we are backing up the database prior to the upgrade so maybe it is possible to search for this backup and getting an older version of your DB. > > If the DB is deleted and you have no backups you will probably have to reinstall. > I recommend to backup your DB on a daily basis using our backup.sh script , you can use a simple cron job for that. > > Eli > > >> >> Thanks. >> >> Regards. >> >> Neil. >> >> >> ---------- Forwarded message ---------- >> From: Neil <nwilson123(a)gmail.com&gt; >> Date: Mon, Oct 15, 2012 at 9:10 AM >> Subject: database issue after power failure >> To: users(a)ovirt.org >> >> >> Hi guys, >> >> I've got what looks like a huge issue after a power failure. >> >> In my server.log I'm seeing "Caused by: >> org.postgresql.util.PSQLException: FATAL: database "engine" does not >> exist" in my server.log and in my engine.log I'm seeing the following >> which also looks very bad. >> >> "2012-10-15 09:08:41,357 ERROR [org.ovirt.engine.core.bll.Backend] >> (MSC service thread 1-4) Error in getting DB connection. The database >> is inaccessible. Original exception is: >> DataAccessResourceFailureException: Error retreiving database >> metadata; nested exception is >> org.springframework.jdbc.support.MetaDataAccessException: Could not >> get Connection for extracting meta data; nested exception is >> org.springframework.jdbc.CannotGetJdbcConnectionException: Could not >> get JDBC Connection; nested exception is java.sql.SQLException: >> javax.resource.ResourceException: IJ000453: Unable to get managed >> connection for java:/ENGINEDataSource" >> >> Please could someone assist if possible, I'm currently logged in on >> #ovirt as well. >> >> Thanks. >> >> Regards. >> >> Neil Wilson. Hi Eli, Thanks for coming back to me. Unfortunately it wasn't an upgrade or installation, just raw db files were removed(rm -f /var/lib/pgsql/*) mid last week, and then there was power failure over the weekend and everything shut down and then refused to start. I've tried running extundelete but it says the "space has been re-allocated" for the files under /var/lib/pgsql The images should in theory still be fine because they are stored on a FC SAN, but I now need to somehow access the .ovf files, back them up, re-install the ovirt-engine from scratch re-initialise the storage and then import the .ovf files. Anyone know how to access the .ovf files without having an engine? I can access the LUNS directly, but can't mount them because of the filesystem type being unknown. Thanks. Regards. Neil.

Look at the post about converting VMs from KVM to oVirt, I think it will help you. Alex 2012/10/15 Neil <nwilson123(a)gmail.com&gt; > > I'm able to access the LUNS directly and I can access the guest LV > filesystems, so I'm currently dumping the guest LV's to raw disk > images, does anyone know how to import these again into ovirt? > > On Mon, Oct 15, 2012 at 11:49 AM, Neil <nwilson123(a)gmail.com&gt; wrote: > > On Mon, Oct 15, 2012 at 11:43 AM, Eli Mesika <emesika(a)redhat.com&gt; wrote: > >> > >> > >> ----- Original Message ----- > >>> From: "Neil" <nwilson123(a)gmail.com&gt; > >>> To: users(a)ovirt.org > >>> Sent: Monday, October 15, 2012 9:27:24 AM > >>> Subject: [Users] Fwd: database issue after power failure > >>> > >>> I've found out now that another technician accidentally deleted my > >>> psql engine db because he tried to install bacula on the same server. > >>> I have no backup of the DB because I was still in the process of > >>> setting them up and wasn't aware he was going to be installing onto > >>> this server. > >>> > >>> I currently have 6 guests that are in a shutdown state because of a > >>> power failure over the weekend. It's using FC storage, only the db > >>> seems to be missing. > >>> > >>> Is there anything I can do to recover from this disaster? > >> > >> Hi > >> Is this a new installation or one that was upgraded? > >> If it is an upgrade IIRC we are backing up the database prior to the > >> upgrade so maybe it is possible to search for this backup and getting an > >> older version of your DB. > >> > >> If the DB is deleted and you have no backups you will probably have to > >> reinstall. > >> I recommend to backup your DB on a daily basis using our backup.sh > >> script , you can use a simple cron job for that. > >> > >> Eli > >> > >> > >>> > >>> Thanks. > >>> > >>> Regards. > >>> > >>> Neil. > >>> > >>> > >>> ---------- Forwarded message ---------- > >>> From: Neil <nwilson123(a)gmail.com&gt; > >>> Date: Mon, Oct 15, 2012 at 9:10 AM > >>> Subject: database issue after power failure > >>> To: users(a)ovirt.org > >>> > >>> > >>> Hi guys, > >>> > >>> I've got what looks like a huge issue after a power failure. > >>> > >>> In my server.log I'm seeing "Caused by: > >>> org.postgresql.util.PSQLException: FATAL: database "engine" does not > >>> exist" in my server.log and in my engine.log I'm seeing the following > >>> which also looks very bad. > >>> > >>> "2012-10-15 09:08:41,357 ERROR [org.ovirt.engine.core.bll.Backend] > >>> (MSC service thread 1-4) Error in getting DB connection. The database > >>> is inaccessible. Original exception is: > >>> DataAccessResourceFailureException: Error retreiving database > >>> metadata; nested exception is > >>> org.springframework.jdbc.support.MetaDataAccessException: Could not > >>> get Connection for extracting meta data; nested exception is > >>> org.springframework.jdbc.CannotGetJdbcConnectionException: Could not > >>> get JDBC Connection; nested exception is java.sql.SQLException: > >>> javax.resource.ResourceException: IJ000453: Unable to get managed > >>> connection for java:/ENGINEDataSource" > >>> > >>> Please could someone assist if possible, I'm currently logged in on > >>> #ovirt as well. > >>> > >>> Thanks. > >>> > >>> Regards. > >>> > >>> Neil Wilson. > > > > Hi Eli, > > > > Thanks for coming back to me. > > > > Unfortunately it wasn't an upgrade or installation, just raw db files > > were removed(rm -f /var/lib/pgsql/*) mid last week, and then there was > > power failure over the weekend and everything shut down and then > > refused to start. > > > > I've tried running extundelete but it says the "space has been > > re-allocated" for the files under /var/lib/pgsql > > > > The images should in theory still be fine because they are stored on a > > FC SAN, but I now need to somehow access the .ovf files, back them up, > > re-install the ovirt-engine from scratch re-initialise the storage and > > then import the .ovf files. > > > > Anyone know how to access the .ovf files without having an engine? I > > can access the LUNS directly, but can't mount them because of the > > filesystem type being unknown. > > > > Thanks. > > > > Regards. > > > > Neil.