Re: [Users] Fedora upgrading from 3.1 to 3.2

From: "Karli Sjöberg" <Karli.Sjoberg(a)slu.se&gt; To: "Alon Bar-Lev" <alonbl(a)redhat.com&gt; Cc: "Alex Lourie" <alourie(a)redhat.com&gt;, users(a)ovirt.org Sent: Thursday, August 1, 2013 10:27:02 AM Subject: Re: [Users] Fedora upgrading from 3.1 to 3.2 ons 2013-07-31 klockan 15:11 -0400 skrev Alon Bar-Lev: ----- Original Message ----- > From: "Karli Sjöberg" <Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se>> > To: "Alex Lourie" <alourie@redhat.com<mailto:alourie@redhat.com>> > Cc: users@ovirt.org<mailto:users@ovirt.org> > Sent: Wednesday, July 31, 2013 5:10:24 PM > Subject: Re: [Users] Fedora upgrading from 3.1 to 3.2 > > Hey! > > Big news for me, I´ve managed to upgrade to 3.2, but sadly, now I can´t get > it to log in with admin@internal :( This is how I managed it: > > (After Fedora is up to 18) > - Erase all of the 3.1-packages > - Install the new 3.2-packages > - Have to go about the dbase upgrade like this: > # psql -U postgres engine > $ ALTER FUNCTION uuid_generate_v1() OWNER TO engine; > Or the upgrade won´t work. After that it´s possible to: > # cd /usr/share/ovirt-engine/dbscripts > # ./upgrade.sh -s localhost -p 5432 -u engine -d engine > (lots of output) > # engine-cleanup --dont-drop- db (not dbase) > > - Had to make modifications to "/usr/bin/engine-setup" too: > @@ -1142,8 +1142,8 @@ > # Run db backup > utils.backupDB( > db=basedefs.DB_NAME, > - backup_file=dbBackupFile, > - env=utils.getPgPassEnv(), > + backupFile=dbBackupFile, > + #env=utils.getPgPassEnv(), > user=getDbUser(), > host=getDbHostName(), > port=getDbPort(), > > An apparent spelling error and an oversight maybe, the "backupDB"-function > in > "common-utils.py" doesn´t expect a "env"-variable so I just uncommented it. > After that it was OK to: > # engine-setup > > No errors. However I was unable to log in so I did: > # systemctl restart ovirt-engine; tail -f /var/log/ovirt-engine/engine.log > | > tee /tmp/engine_start.log > > Where I was able to see stuff like: > "2013-07-31 15:19:13,300 ERROR > [org.ovirt.engine.core.utils.crypt.EncryptionUtils] (MSC service thread > 1-8) > Can't load keystore from file "/keys/engine.p12". FileNotFoundException: > keys/engine.p12 (No such file or directory)" Yes. There was a bug in tools which ignored the CABaseDirectory option. Please execute the following within engine database: update vdc_options set option_value = '/etc/pki/ovirt-engine/keys/engine.p12' where option_name = 'keystoreUrl'; update vdc_options set option_value = '/etc/pki/ovirt-engine/certs/engine.cer' where option_name = 'CertificateFileName'; update vdc_options set option_value = '/etc/pki/ovirt-engine/.truststore' where option_name = 'TruststoreUrl'; OK, done, thanks! Now I just get this printed over and over: 2013-08-01 09:09:04,929 ERROR [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-3) Error in getting DB connection. The database is inaccessible. Original exception is: DataAccessResourceFailureException: Error retreiving database metadata; nested exception is org.springframework.jdbc.support.MetaDataAccessException: Could not get Connection for extracting meta data; nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource What now?

> > Although the file does exist (freshly created during "engine-setup") and is > located exactly where it should be, in "/etc/pki/ovirt-engine/keys". > > Files attached. > > Any ideas? > > /Karli > > tor 2013-07-18 klockan 10:02 +0000 skrev Karli Sjöberg: > > > Hi Alex, > > since I´m on vacation I haven´t been as active as I usually am, but I > haven´t > stopped working. Good to hear about the certificates, and about > "--dont-drop-database". I´m going to contact one of my co-workers and have > him try the folowing scenario: > > 1) Have a working F17/3.1 system up and running > 2) Use "fedora-upgrade" up to F18 > 3) "yum erase" all oVirt-packages > 4) Install the latest 3.2-packages > 5) Execute: > # cd /usr/share/ovirt-engine/dbscripts > # ./upgrade.sh -s localhost -p 5432 -u engine -d engine > # engine-cleanup --dont-drop-database > 6) "engine-setup" > > And see how that goes. First I´m thinking about testing with a "clean", > just > installed engine with essentially nothing in it. And if the procedure > proves > successful, try again from the start but with an engine that has Hosts, > Storage, VMs and so on added into it. > > /Karli > > ________________________________________ > Från: Alex Lourie [ alourie@redhat.com<mailto:alourie@redhat.com> ] > Skickat: den 14 juli 2013 13:42 > Till: Karli Sjöberg > Cc: users@ovirt.org<mailto:users@ovirt.org> Ämne: Re: [Users] Fedora > upgrading from 3.1 to 3.2 > > Hi Karli > > It's been awhile since we heard from you. Would you mind updating us on > your > status? > > Thanks > > -- > > Best regards, > > Alex Lourie > Software Developer in Integration > Red Hat > > > ----- Original Message ----- > > From: "Alex Lourie" < alourie@redhat.com<mailto:alourie@redhat.com> > > > To: "Karli Sjöberg" < Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se> > > > Cc: users@ovirt.org<mailto:users@ovirt.org> > Sent: Wednesday, July 10, > > 2013 7:16:02 PM > > Subject: Re: [Users] Fedora upgrading from 3.1 to 3.2 > > > > Hi Karli > > > > 'Restore' certificates basically means taking the backup of > > /etc/pki/ovirt-engine/certs and /keys and restoring them into 3.2 after > > installation. > > > > --dont-drop-database will do exactly that - leave DB intact; that can be > > for > > your benefit in some cases. > > > > I'll be happy to hear on your progress. > > > > -- > > > > Best regards, > > > > Alex Lourie > > Software Developer in Integration > > Red Hat > > > > > > ----- Original Message ----- > > > From: "Karli Sjöberg" < > > > Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se> > > > > To: "Alex Lourie" < alourie@redhat.com<mailto:alourie@redhat.com> > > > > Cc: users@ovirt.org<mailto:users@ovirt.org> > > Sent: Friday, July 5, > > > 2013 8:34:22 PM > > > Subject: SV: [Users] Fedora upgrading from 3.1 to 3.2 > > > > > > Hi Alex, > > > > > > crappy MS webmail that can´t figure out indents while on vacation, just > > > FYI. > > > Yes, progress is always good:) I would like to have some pointers about > > > nr. > > > 4: Restore certificates. Then I can ask one of my co-workers to test > > > the > > > procedure and report back. So, restore certs; How to? > > > > > > Or! I saw in another thread there was a "engine-cleanup > > > --dont-drop-dbase" > > > something or other... Is there an equivalent for "engine-setup", like > > > "--dont-touch-dbase"? Or "engine-cleanup --dbase-something" and then > > > "engine-setup" again, and it´ll just play nice with the dbase that´s > > > still > > > there perhaps? > > > > > > /Karli > > > > > > ________________________________________ > > > Från: Alex Lourie [ alourie@redhat.com<mailto:alourie@redhat.com> ] > > > Skickat: den 30 juni 2013 17:29 > > > Till: Karli Sjöberg > > > Cc: users@ovirt.org<mailto:users@ovirt.org> > > Ämne: Re: [Users] > > > Fedora upgrading from 3.1 to > > > 3.2 > > > > > > Hi Karli > > > > > > On Wed, Jun 26, 2013 at 5:28 PM, Karli Sjöberg < > > > Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se> > > > > wrote: > > > > Update! > > > > > > > > I have actually made some headway, I managed to get it passed the > > > > database upgrade! As I was looking at the log, I drilled down into > > > > the engine-upgrade script at looked at what it was trying to do, > > > > which was a pythonic way of doing: > > > > # cd /usr/share/ovirt-engine/dbscripts > > > > # ./upgrade.sh -s localhost -p 5432 -u engine -d engine > > > > > > > > > > Nice progress! > > > > > > > > > > > So I first reinstalled everything (I´ve stopped counting) and an > > > > hour later I was back at just before doing engine-upgrade again. What > > > > I did then was to upgrade ovirt-engine-dbscripts first and ran the > > > > upgrade.sh script manually (which went by smoothly, output also > > > > attached), downgraded all of the ovirt packages back to 3.1.0-4 > > > > (because engine-upgrade didn´t think there was any updating to act > > > > upon otherwise), then updated ovirt-engine-setup to 3.2.2, lastly ran > > > > engine-upgrade. That made it pass the database upgrade, yeay! But! > > > > Now it stopped at doing the CA instead... Log is attached. > > > > > > > > What I find strange is the last line of output from the upgrade.sh: > > > > ... > > > > Refreshing materialized views... > > > > Done. > > > > /usr/share/ovirt-engine/dbscripts > > > > > > > > Where the last lines of upgrade.sh are like: > > > > ... > > > > run_upgrade_files > > > > > > > > ret=$? > > > > printf "Done.\n" > > > > exit $ret > > > > > > > > And I looked through run_upgrade_files and couldn´t figure out why > > > > that function would exit with "/usr/share/ovirt-engine/dbscripts"? No > > > > that doesn´t quite add up. Something just doesn´t smell right but I > > > > haven´t figured it out what it is yet, > > > > > > > > > > I think it is the output from another script that runs the upgrade.sh > > > script. There's a pushd/popd action used in sh scripts to change > > > working folder and then get back to the original one. I think that this > > > is what you see here. > > > > > > > > > > > About the second error, with handling the CA, it seems like it´s > > > > having trouble connecting to the database after it´s upgrade, but > > > > since the upgrade itself went by OK, I did engine-cleanup, > > > > engine-setup, stopped engine, restored the database and started > > > > engine again, and it worked. Which should point more to something > > > > like a configuration being missed, or improperly handled at the > > > > upgrade that makes engine-config fail to connect. > > > > > > Could be; but the fact that reimport of the DB worked is a very good > > > sign. > > > > > > > Maybe some old configurations lying around that shouldn´t be, that > > > > hinders it, I don´t know. Is there anything handling configuration > > > > changes, like "rpmconf -a" in the upgrade process, after updating the > > > > rpms? > > > > > > > > > > No, we do not have rpmconf in ovirt. > > > > > > Now, if you have a working engine, that's good. Remember though that > > > you started with clean DB. When DB has entries, the upgrade.sh run may > > > not be as fluent as it was before. But, if it works, I guess you could > > > use the same strategy for upgrading the complete working environment: > > > > > > 1. Backup DB and certificates > > > 2. Upgrade DB in a stand-alone mode and save it aside. > > > 3. Install 3.2, restore the engine DB from upgraded backup. > > > 4. Restore certificates. > > > 5. Start the engine. > > > 6. Enjoy? > > > > > > Let me know what your status is. > > > > > > > > > > > > > > /Karli > > > > > > > > ons 2013-06-26 klockan 13:33 +0000 skrev Karli Sjöberg: > > > >> tis 2013-06-25 klockan 16:02 +0003 skrev Alex Lourie:  > > > >>> On Tue, Jun 25, 2013 at 5:03 PM, Karli Sjöberg > > > >>> < Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se> > > > > >>> wrote: > > > >>> > tis 2013-06-25 klockan 15:35 +0200 skrev Gianluca Cecchi: On > > > >>> Tue, > > > >>> > Jun 25, 2013 at 2:58 PM, Karli Sjöberg wrote: > > > >>> >  > > > >>> >>> > > > >>> >>> > > > >>> > Yes, I had much better success following that article, and > > > >>> managed > > > >>> > to upgrade fedora to 18, had to tune my kernel parameters a > > > >>> little > > > >>> > for the postgres upgrade to work, but then engine-upgrade fails > > > >>> just > > > >>> > as it did the last time we tried. The log is attached. Hoping to > > > >>> hear > > > >>> > back from you soon with ideas on what to try next. > > > >>> >  > > > >>> >>> > > > >>> >>> > > > >>> >>> > > > >>> >  > > > >>> >> > > > >>> >> > > > >>> > But now the error seems different from the original one in > > > >>> March > > > >>> > (if I remember correctly the original post). > > > >>> >> > > > >>> > > > > >>> > They are quite the same: > > > >>> > > > > >>> > Then > > > >>> > psql:drop_old_uuid_functions.sql:25: ERROR: cannot drop > > > >>> function > > > >>> > uuid_nil() because extension uuid-ossp requires it > > > >>> > > > > >>> > Now > > > >>> > 2013-06-25 14:52:16::DEBUG::common_utils::473::root:: stderr = > > > >>> > psql:drop_old_uuid_functions.sql:25: ERROR: cannot drop > > > >>> function > > > >>> > uuid_nil() because extension uuid-ossp requires it > > > >>> > > > > >>> >> Could it be somehow related with this rhev one bug: > > > >>> >  https://bugzilla.redhat.com/show_bug.cgi?id=923614 > > >>> >> > > > >>> >> > > > >>> > and that changing permissions for the impacted objects in db > > > >>> before > > > >>> > running upgrade could help? > > > >>> > > > >> > > > >> I have now tried that and it did no change in success there > > > >> unfortunately, so it´s probably not anything permissions related in > > > >> the database. > > > >> > > > >> dbmodify: > > > >> #!/bin/sh > > > >> export PGPASSFILE=/root/.pgpass > > > >> for tbl in `psql -U postgres -wqAt -c "select tablename from > > > >> pg_tables where schemaname = 'public';" engine` ; do psql -U > > > >> postgres -c "alter table $tbl owner to engine" engine ; done > > > >> for tbl in `psql -U postgres -wqAt -c "select sequence_name from > > > >> information_schema.sequences where sequence_schema = 'public';" > > > >> engine`; do psql -U postgres -c "alter table $tbl owner to engine" > > > >> engine ; done > > > >> for tbl in `psql -U postgres -wqAt -c "select table_name from > > > >> information_schema.views where table_schema = 'public';" engine` ; > > > >> do psql -U postgres -c "alter table $tbl owner to engine" engine ; > > > >> done > > > >> > > > >> # yum update -y ovirt-engine-setup > > > >> # ./dbmodify > > > >> # engine-upgrade > > > >> > > > >> fail > > > >> > > > >> Log is attached this time as well, it bombs out at the exact same > > > >> place, in the same way. > > > >> > > > >> /K > > > >> > > > >>> >> > > > >>> > > > > >>> > Very interesting, thanks for the hint! That is definitely worth > > > >>> > trying out. Will try that first thing tomorrow:) > > > >>> > > > >>> Hi Karli > > > >>> > > > >>> If it helps, let me know - I will add a troubleshooting section to > > > >>> the > > > >>> wiki. > > > >>> > > > >>> > > > > >>> > > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> > BTW: is your environment directly created in 3.1 or did it come > > > >>> > from a further upgrade? > > > >>> >> > > > >>> > > > > >>> > Started with a minimal Fedora 17 install, added the oVirt-3.1 > > > >>> repo, > > > >>> > ran yum upgrade -y first, then yum install -y ovirt-engine and > > > >>> lastly > > > >>> > engine-setup. Nothing more. So it´s just an empty engine that > > > >>> I´m > > > >>> > trying to upgrade at this point. When that works, I´ll add > > > >>> hosts, > > > >>> > VMs, Templates, etc. > > > >>> > > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> > HIH, > > > >>> > Gianluca > > > >>> > > > > >>> > -- > > > >>> > > > > >>> > Med Vänliga Hälsningar > > > >>> > > > > >>> ------------------------------------------------------------------------------- > > > >>> > Karli Sjöberg > > > >>> > Swedish University of Agricultural Sciences > > > >>> > Box 7079 (Visiting Address Kronåsvägen 8) > > > >>> > S-750 07 Uppsala, Sweden > > > >>> > Phone: +46-(0)18-67 15 66 > > > >>> > karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se> > > >>> > > > >>> > > > >> > > > >> -- > > > >> > > > >> Med Vänliga Hälsningar > > > >> ------------------------------------------------------------------------------- > > > >> Karli Sjöberg > > > >> Swedish University of Agricultural Sciences > > > >> Box 7079 (Visiting Address Kronåsvägen 8) > > > >> S-750 07 Uppsala, Sweden > > > >> Phone: +46-(0)18-67 15 66 > > > >> karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se> > > > > > > > -- > > > > > > > > Med Vänliga Hälsningar > > > > ------------------------------------------------------------------------------- > > > > Karli Sjöberg > > > > Swedish University of Agricultural Sciences > > > > Box 7079 (Visiting Address Kronåsvägen 8) > > > > S-750 07 Uppsala, Sweden > > > > Phone: +46-(0)18-67 15 66 > > > > karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se> > > > > > > > _______________________________________________ > > Users mailing list > > Users@ovirt.org<mailto:Users@ovirt.org> > > > http://lists.ovirt.org/mailman/listinfo/users > > _______________________________________________ > Users mailing list Users@ovirt.org<mailto:Users@ovirt.org> > http://lists.ovirt.org/mailman/listinfo/users > > > -- > > Med Vänliga Hälsningar > ------------------------------------------------------------------------------- > Karli Sjöberg > Swedish University of Agricultural Sciences > Box 7079 (Visiting Address Kronåsvägen 8) > S-750 07 Uppsala, Sweden > Phone: +46-(0)18-67 15 66 > karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se> > > _______________________________________________ > Users mailing list > Users@ovirt.org<mailto:Users@ovirt.org> > http://lists.ovirt.org/mailman/listinfo/users > -- Med Vänliga Hälsningar ------------------------------------------------------------------------------- Karli Sjöberg Swedish University of Agricultural Sciences Box 7079 (Visiting Address Kronåsvägen 8) S-750 07 Uppsala, Sweden Phone: +46-(0)18-67 15 66 karli.sjoberg@slu.se<mailto:karli.sjoberg@adm.slu.se>