Re: [Users] Fedora upgrading from 3.1 to 3.2

From: "Karli Sjöberg" <Karli.Sjoberg(a)slu.se&gt; To: "Alon Bar-Lev" <alonbl(a)redhat.com&gt; Cc: "Alex Lourie" <alourie(a)redhat.com&gt;, users(a)ovirt.org, "Eli Mesika" <emesika(a)redhat.com&gt; Sent: Thursday, August 1, 2013 11:02:13 AM Subject: Re: [Users] Fedora upgrading from 3.1 to 3.2 tor 2013-08-01 klockan 03:44 -0400 skrev Alon Bar-Lev: ----- Original Message ----- > From: "Karli Sjöberg" <Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se>> > To: "Alon Bar-Lev" <alonbl@redhat.com<mailto:alonbl@redhat.com>> > Cc: "Alex Lourie" <alourie@redhat.com<mailto:alourie@redhat.com>>, > users@ovirt.org<mailto:users@ovirt.org> > Sent: Thursday, August 1, 2013 10:27:02 AM > Subject: Re: [Users] Fedora upgrading from 3.1 to 3.2 > > ons 2013-07-31 klockan 15:11 -0400 skrev Alon Bar-Lev: > > > > ----- Original Message ----- > > From: "Karli Sjöberg" > > <Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se><mailto:Karli.Sjoberg@slu.se>> > > To: "Alex Lourie" > > <alourie@redhat.com<mailto:alourie@redhat.com><mailto:alourie@redhat.com>> > > Cc: users@ovirt.org<mailto:users@ovirt.org><mailto:users@ovirt.org> > > Sent: Wednesday, July 31, 2013 5:10:24 PM > > Subject: Re: [Users] Fedora upgrading from 3.1 to 3.2 > > > > Hey! > > > > Big news for me, I´ve managed to upgrade to 3.2, but sadly, now I can´t > > get > > it to log in with admin@internal :( This is how I managed it: > > > > (After Fedora is up to 18) > > - Erase all of the 3.1-packages > > - Install the new 3.2-packages > > - Have to go about the dbase upgrade like this: > > # psql -U postgres engine > > $ ALTER FUNCTION uuid_generate_v1() OWNER TO engine; > > Or the upgrade won´t work. After that it´s possible to: > > # cd /usr/share/ovirt-engine/dbscripts > > # ./upgrade.sh -s localhost -p 5432 -u engine -d engine > > (lots of output) > > # engine-cleanup --dont-drop- db (not dbase) > > > > - Had to make modifications to "/usr/bin/engine-setup" too: > > @@ -1142,8 +1142,8 @@ > > # Run db backup > > utils.backupDB( > > db=basedefs.DB_NAME, > > - backup_file=dbBackupFile, > > - env=utils.getPgPassEnv(), > > + backupFile=dbBackupFile, > > + #env=utils.getPgPassEnv(), > > user=getDbUser(), > > host=getDbHostName(), > > port=getDbPort(), > > > > An apparent spelling error and an oversight maybe, the > > "backupDB"-function > > in > > "common-utils.py" doesn´t expect a "env"-variable so I just uncommented > > it. > > After that it was OK to: > > # engine-setup > > > > No errors. However I was unable to log in so I did: > > # systemctl restart ovirt-engine; tail -f > > /var/log/ovirt-engine/engine.log > > | > > tee /tmp/engine_start.log > > > > Where I was able to see stuff like: > > "2013-07-31 15:19:13,300 ERROR > > [org.ovirt.engine.core.utils.crypt.EncryptionUtils] (MSC service thread > > 1-8) > > Can't load keystore from file "/keys/engine.p12". FileNotFoundException: > > keys/engine.p12 (No such file or directory)" > > Yes. > There was a bug in tools which ignored the CABaseDirectory option. > Please execute the following within engine database: > > update vdc_options set option_value = > '/etc/pki/ovirt-engine/keys/engine.p12' > where option_name = 'keystoreUrl'; > update vdc_options set option_value = > '/etc/pki/ovirt-engine/certs/engine.cer' where option_name = > 'CertificateFileName'; > update vdc_options set option_value = '/etc/pki/ovirt-engine/.truststore' > where option_name = 'TruststoreUrl'; > > > OK, done, thanks! Now I just get this printed over and over: > 2013-08-01 09:09:04,929 ERROR [org.ovirt.engine.core.bll.Backend] (MSC > service thread 1-3) Error in getting DB connection. The database is > inaccessible. Original exception is: DataAccessResourceFailureException: > Error retreiving database metadata; nested exception is > org.springframework.jdbc.support.MetaDataAccessException: Could not get > Connection for extracting meta data; nested exception is > org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get > JDBC Connection; nested exception is java.sql.SQLException: > javax.resource.ResourceException: IJ000453: Unable to get managed > connection > for java:/ENGINEDataSource > > What now? > This I do not know... What do you have at: Well, essentially nothing... /etc/ovirt-engine/.pgpass # This section was created during oVirt Engine setup. # DO NOT CHANGE IT MANUALLY - OTHER UTILITIES AND TOOLS DEPEND ON ITS STRUCTURE. # Beginning of the oVirt Engine DB settings section # DB ADMIN credentials. localhost:5432:*:postgres:letmein! # DB USER credentials. localhost:5432:*:engine:letmein! ##### End of oVirt Engine DB settings section. /etc/sysconfig/ovirt-engine # # For descriptions of the parameters and their default values look # at the /usr/share/ovirt-engine/conf/engine.conf.defaults file. # # Please note that the engine installation tool engine-setup will # append the modified parameters to the end of this file. #

/Karli > > > > > > > Although the file does exist (freshly created during "engine-setup") and > > is > > located exactly where it should be, in "/etc/pki/ovirt-engine/keys". > > > > Files attached. > > > > Any ideas? > > > > /Karli > > > > tor 2013-07-18 klockan 10:02 +0000 skrev Karli Sjöberg: > > > > > > Hi Alex, > > > > since I´m on vacation I haven´t been as active as I usually am, but I > > haven´t > > stopped working. Good to hear about the certificates, and about > > "--dont-drop-database". I´m going to contact one of my co-workers and > > have > > him try the folowing scenario: > > > > 1) Have a working F17/3.1 system up and running > > 2) Use "fedora-upgrade" up to F18 > > 3) "yum erase" all oVirt-packages > > 4) Install the latest 3.2-packages > > 5) Execute: > > # cd /usr/share/ovirt-engine/dbscripts > > # ./upgrade.sh -s localhost -p 5432 -u engine -d engine > > # engine-cleanup --dont-drop-database > > 6) "engine-setup" > > > > And see how that goes. First I´m thinking about testing with a "clean", > > just > > installed engine with essentially nothing in it. And if the procedure > > proves > > successful, try again from the start but with an engine that has Hosts, > > Storage, VMs and so on added into it. > > > > /Karli > > > > ________________________________________ > > Från: Alex Lourie [ > > alourie@redhat.com<mailto:alourie@redhat.com><mailto:alourie@redhat.com> > > ] > > Skickat: den 14 juli 2013 13:42 > > Till: Karli Sjöberg > > Cc: users@ovirt.org<mailto:users@ovirt.org><mailto:users@ovirt.org> Ämne: > > Re: [Users] Fedora > > upgrading from 3.1 to 3.2 > > > > Hi Karli > > > > It's been awhile since we heard from you. Would you mind updating us on > > your > > status? > > > > Thanks > > > > -- > > > > Best regards, > > > > Alex Lourie > > Software Developer in Integration > > Red Hat > > > > > > ----- Original Message ----- > > > From: "Alex Lourie" < > > > alourie@redhat.com<mailto:alourie@redhat.com><mailto:alourie@redhat.com> > > > > > > > To: "Karli Sjöberg" < > > > Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se><mailto:Karli.Sjoberg@slu.se> > > > > > > > Cc: users@ovirt.org<mailto:users@ovirt.org><mailto:users@ovirt.org> > > > > Sent: Wednesday, July 10, > > > 2013 7:16:02 PM > > > Subject: Re: [Users] Fedora upgrading from 3.1 to 3.2 > > > > > > Hi Karli > > > > > > 'Restore' certificates basically means taking the backup of > > > /etc/pki/ovirt-engine/certs and /keys and restoring them into 3.2 after > > > installation. > > > > > > --dont-drop-database will do exactly that - leave DB intact; that can > > > be > > > for > > > your benefit in some cases. > > > > > > I'll be happy to hear on your progress. > > > > > > -- > > > > > > Best regards, > > > > > > Alex Lourie > > > Software Developer in Integration > > > Red Hat > > > > > > > > > ----- Original Message ----- > > > > From: "Karli Sjöberg" < > > > > Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se><mailto:Karli.Sjoberg@slu.se> > > > > > > > > > To: "Alex Lourie" < > > > > alourie@redhat.com<mailto:alourie@redhat.com><mailto:alourie@redhat.com> > > > > > > > > > Cc: users@ovirt.org<mailto:users@ovirt.org><mailto:users@ovirt.org> > > > > > > Sent: Friday, July 5, > > > > 2013 8:34:22 PM > > > > Subject: SV: [Users] Fedora upgrading from 3.1 to 3.2 > > > > > > > > Hi Alex, > > > > > > > > crappy MS webmail that can´t figure out indents while on vacation, > > > > just > > > > FYI. > > > > Yes, progress is always good:) I would like to have some pointers > > > > about > > > > nr. > > > > 4: Restore certificates. Then I can ask one of my co-workers to test > > > > the > > > > procedure and report back. So, restore certs; How to? > > > > > > > > Or! I saw in another thread there was a "engine-cleanup > > > > --dont-drop-dbase" > > > > something or other... Is there an equivalent for "engine-setup", like > > > > "--dont-touch-dbase"? Or "engine-cleanup --dbase-something" and then > > > > "engine-setup" again, and it´ll just play nice with the dbase that´s > > > > still > > > > there perhaps? > > > > > > > > /Karli > > > > > > > > ________________________________________ > > > > Från: Alex Lourie [ > > > > alourie@redhat.com<mailto:alourie@redhat.com><mailto:alourie@redhat.com> > > > > ] > > > > Skickat: den 30 juni 2013 17:29 > > > > Till: Karli Sjöberg > > > > Cc: users@ovirt.org<mailto:users@ovirt.org><mailto:users@ovirt.org> > > > > > > Ämne: Re: [Users] > > > > Fedora upgrading from 3.1 to > > > > 3.2 > > > > > > > > Hi Karli > > > > > > > > On Wed, Jun 26, 2013 at 5:28 PM, Karli Sjöberg < > > > > Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se><mailto:Karli.Sjoberg@slu.se> > > > > > > > > > wrote: > > > > > Update! > > > > > > > > > > I have actually made some headway, I managed to get it passed the > > > > > database upgrade! As I was looking at the log, I drilled down into > > > > > the engine-upgrade script at looked at what it was trying to do, > > > > > which was a pythonic way of doing: > > > > > # cd /usr/share/ovirt-engine/dbscripts > > > > > # ./upgrade.sh -s localhost -p 5432 -u engine -d engine > > > > > > > > > > > > > Nice progress! > > > > > > > > > > > > > > So I first reinstalled everything (I´ve stopped counting) and an > > > > > hour later I was back at just before doing engine-upgrade again. > > > > > What > > > > > I did then was to upgrade ovirt-engine-dbscripts first and ran the > > > > > upgrade.sh script manually (which went by smoothly, output also > > > > > attached), downgraded all of the ovirt packages back to 3.1.0-4 > > > > > (because engine-upgrade didn´t think there was any updating to act > > > > > upon otherwise), then updated ovirt-engine-setup to 3.2.2, lastly > > > > > ran > > > > > engine-upgrade. That made it pass the database upgrade, yeay! But! > > > > > Now it stopped at doing the CA instead... Log is attached. > > > > > > > > > > What I find strange is the last line of output from the upgrade.sh: > > > > > ... > > > > > Refreshing materialized views... > > > > > Done. > > > > > /usr/share/ovirt-engine/dbscripts > > > > > > > > > > Where the last lines of upgrade.sh are like: > > > > > ... > > > > > run_upgrade_files > > > > > > > > > > ret=$? > > > > > printf "Done.\n" > > > > > exit $ret > > > > > > > > > > And I looked through run_upgrade_files and couldn´t figure out why > > > > > that function would exit with "/usr/share/ovirt-engine/dbscripts"? > > > > > No > > > > > that doesn´t quite add up. Something just doesn´t smell right but I > > > > > haven´t figured it out what it is yet, > > > > > > > > > > > > > I think it is the output from another script that runs the upgrade.sh > > > > script. There's a pushd/popd action used in sh scripts to change > > > > working folder and then get back to the original one. I think that > > > > this > > > > is what you see here. > > > > > > > > > > > > > > About the second error, with handling the CA, it seems like it´s > > > > > having trouble connecting to the database after it´s upgrade, but > > > > > since the upgrade itself went by OK, I did engine-cleanup, > > > > > engine-setup, stopped engine, restored the database and started > > > > > engine again, and it worked. Which should point more to something > > > > > like a configuration being missed, or improperly handled at the > > > > > upgrade that makes engine-config fail to connect. > > > > > > > > Could be; but the fact that reimport of the DB worked is a very good > > > > sign. > > > > > > > > > Maybe some old configurations lying around that shouldn´t be, that > > > > > hinders it, I don´t know. Is there anything handling configuration > > > > > changes, like "rpmconf -a" in the upgrade process, after updating > > > > > the > > > > > rpms? > > > > > > > > > > > > > No, we do not have rpmconf in ovirt. > > > > > > > > Now, if you have a working engine, that's good. Remember though that > > > > you started with clean DB. When DB has entries, the upgrade.sh run > > > > may > > > > not be as fluent as it was before. But, if it works, I guess you > > > > could > > > > use the same strategy for upgrading the complete working environment: > > > > > > > > 1. Backup DB and certificates > > > > 2. Upgrade DB in a stand-alone mode and save it aside. > > > > 3. Install 3.2, restore the engine DB from upgraded backup. > > > > 4. Restore certificates. > > > > 5. Start the engine. > > > > 6. Enjoy? > > > > > > > > Let me know what your status is. > > > > > > > > > > > > > > > > > > /Karli > > > > > > > > > > ons 2013-06-26 klockan 13:33 +0000 skrev Karli Sjöberg: > > > > >> tis 2013-06-25 klockan 16:02 +0003 skrev Alex Lourie:  > > > > >>> On Tue, Jun 25, 2013 at 5:03 PM, Karli Sjöberg > > > > >>> < > > > > >>> Karli.Sjoberg@slu.se<mailto:Karli.Sjoberg@slu.se><mailto:Karli.Sjoberg@slu.se> > > > > >>> > > > > > >>> wrote: > > > > >>> > tis 2013-06-25 klockan 15:35 +0200 skrev Gianluca Cecchi: On > > > > >>> Tue, > > > > >>> > Jun 25, 2013 at 2:58 PM, Karli Sjöberg wrote: > > > > >>> >  > > > > >>> >>> > > > > >>> >>> > > > > >>> > Yes, I had much better success following that article, and > > > > >>> managed > > > > >>> > to upgrade fedora to 18, had to tune my kernel parameters a > > > > >>> little > > > > >>> > for the postgres upgrade to work, but then engine-upgrade fails > > > > >>> just > > > > >>> > as it did the last time we tried. The log is attached. Hoping > > > > >>> > to > > > > >>> hear > > > > >>> > back from you soon with ideas on what to try next. > > > > >>> >  > > > > >>> >>> > > > > >>> >>> > > > > >>> >>> > > > > >>> >  > > > > >>> >> > > > > >>> >> > > > > >>> > But now the error seems different from the original one in > > > > >>> March > > > > >>> > (if I remember correctly the original post). > > > > >>> >> > > > > >>> > > > > > >>> > They are quite the same: > > > > >>> > > > > > >>> > Then > > > > >>> > psql:drop_old_uuid_functions.sql:25: ERROR: cannot drop > > > > >>> function > > > > >>> > uuid_nil() because extension uuid-ossp requires it > > > > >>> > > > > > >>> > Now > > > > >>> > 2013-06-25 14:52:16::DEBUG::common_utils::473::root:: stderr = > > > > >>> > psql:drop_old_uuid_functions.sql:25: ERROR: cannot drop > > > > >>> function > > > > >>> > uuid_nil() because extension uuid-ossp requires it > > > > >>> > > > > > >>> >> Could it be somehow related with this rhev one bug: > > > > >>> >  https://bugzilla.redhat.com/show_bug.cgi?id=923614 > > >>> > > > > >>> > >> > > > > >>> >> > > > > >>> > and that changing permissions for the impacted objects in db > > > > >>> before > > > > >>> > running upgrade could help? > > > > >>> > > > > >> > > > > >> I have now tried that and it did no change in success there > > > > >> unfortunately, so it´s probably not anything permissions related > > > > >> in > > > > >> the database. > > > > >> > > > > >> dbmodify: > > > > >> #!/bin/sh > > > > >> export PGPASSFILE=/root/.pgpass > > > > >> for tbl in `psql -U postgres -wqAt -c "select tablename from > > > > >> pg_tables where schemaname = 'public';" engine` ; do psql -U > > > > >> postgres -c "alter table $tbl owner to engine" engine ; done > > > > >> for tbl in `psql -U postgres -wqAt -c "select sequence_name from > > > > >> information_schema.sequences where sequence_schema = 'public';" > > > > >> engine`; do psql -U postgres -c "alter table $tbl owner to > > > > >> engine" > > > > >> engine ; done > > > > >> for tbl in `psql -U postgres -wqAt -c "select table_name from > > > > >> information_schema.views where table_schema = 'public';" engine` ; > > > > >> do psql -U postgres -c "alter table $tbl owner to engine" engine ; > > > > >> done > > > > >> > > > > >> # yum update -y ovirt-engine-setup > > > > >> # ./dbmodify > > > > >> # engine-upgrade > > > > >> > > > > >> fail > > > > >> > > > > >> Log is attached this time as well, it bombs out at the exact same > > > > >> place, in the same way. > > > > >> > > > > >> /K > > > > >> > > > > >>> >> > > > > >>> > > > > > >>> > Very interesting, thanks for the hint! That is definitely worth > > > > >>> > trying out. Will try that first thing tomorrow:) > > > > >>> > > > > >>> Hi Karli > > > > >>> > > > > >>> If it helps, let me know - I will add a troubleshooting section > > > > >>> to > > > > >>> the > > > > >>> wiki. > > > > >>> > > > > >>> > > > > > >>> > > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> > BTW: is your environment directly created in 3.1 or did it > > > > >>> > come > > > > >>> > from a further upgrade? > > > > >>> >> > > > > >>> > > > > > >>> > Started with a minimal Fedora 17 install, added the oVirt-3.1 > > > > >>> repo, > > > > >>> > ran yum upgrade -y first, then yum install -y ovirt-engine and > > > > >>> lastly > > > > >>> > engine-setup. Nothing more. So it´s just an empty engine that > > > > >>> I´m > > > > >>> > trying to upgrade at this point. When that works, I´ll add > > > > >>> hosts, > > > > >>> > VMs, Templates, etc. > > > > >>> > > > > > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> > HIH, > > > > >>> > Gianluca > > > > >>> > > > > > >>> > -- > > > > >>> > > > > > >>> > Med Vänliga Hälsningar > > > > >>> > > > > > >>> ------------------------------------------------------------------------------- > > > > >>> > Karli Sjöberg > > > > >>> > Swedish University of Agricultural Sciences > > > > >>> > Box 7079 (Visiting Address Kronåsvägen 8) > > > > >>> > S-750 07 Uppsala, Sweden > > > > >>> > Phone: +46-(0)18-67 15 66 > > > > >>> > karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se><mailto:karli.sjoberg@slu.se> > > > > >>> > > > >>> > > > > >>> > > > > >> > > > > >> -- > > > > >> > > > > >> Med Vänliga Hälsningar > > > > >> ------------------------------------------------------------------------------- > > > > >> Karli Sjöberg > > > > >> Swedish University of Agricultural Sciences > > > > >> Box 7079 (Visiting Address Kronåsvägen 8) > > > > >> S-750 07 Uppsala, Sweden > > > > >> Phone: +46-(0)18-67 15 66 > > > > >> karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se><mailto:karli.sjoberg@slu.se> > > > > >> > > > > > > > > -- > > > > > > > > > > Med Vänliga Hälsningar > > > > > ------------------------------------------------------------------------------- > > > > > Karli Sjöberg > > > > > Swedish University of Agricultural Sciences > > > > > Box 7079 (Visiting Address Kronåsvägen 8) > > > > > S-750 07 Uppsala, Sweden > > > > > Phone: +46-(0)18-67 15 66 > > > > > karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se><mailto:karli.sjoberg@slu.se> > > > > > > > > > > > > > > _______________________________________________ > > > Users mailing list > > > Users@ovirt.org<mailto:Users@ovirt.org><mailto:Users@ovirt.org> > > > > http://lists.ovirt.org/mailman/listinfo/users > > > _______________________________________________ > > Users mailing list > > Users@ovirt.org<mailto:Users@ovirt.org><mailto:Users@ovirt.org> > > http://lists.ovirt.org/mailman/listinfo/users > > > > > > -- > > > > Med Vänliga Hälsningar > > ------------------------------------------------------------------------------- > > Karli Sjöberg > > Swedish University of Agricultural Sciences > > Box 7079 (Visiting Address Kronåsvägen 8) > > S-750 07 Uppsala, Sweden > > Phone: +46-(0)18-67 15 66 > > karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se><mailto:karli.sjoberg@slu.se> > > > > _______________________________________________ > > Users mailing list > > Users@ovirt.org<mailto:Users@ovirt.org><mailto:Users@ovirt.org> > > http://lists.ovirt.org/mailman/listinfo/users > > > > > -- > > Med Vänliga Hälsningar > ------------------------------------------------------------------------------- > Karli Sjöberg > Swedish University of Agricultural Sciences > Box 7079 (Visiting Address Kronåsvägen 8) > S-750 07 Uppsala, Sweden > Phone: +46-(0)18-67 15 66 > karli.sjoberg@slu.se<mailto:karli.sjoberg@slu.se><mailto:karli.sjoberg@adm.slu.se> > -- Med Vänliga Hälsningar ------------------------------------------------------------------------------- Karli Sjöberg Swedish University of Agricultural Sciences Box 7079 (Visiting Address Kronåsvägen 8) S-750 07 Uppsala, Sweden Phone: +46-(0)18-67 15 66 karli.sjoberg@slu.se<mailto:karli.sjoberg@adm.slu.se>