<div dir="ltr"><div><div><div><div><div><div>Agreed,<br></div><div>I did revert my victim VM back the pre-upgrade snapshot and retry the upgrade with the same results with yesterdays packages.I had watch on the logs but I did not see any DB upgrade failures.<br>
<br>Given that I figured for the heck of it I would just rebuild with this mornings latest commits to see if the issue still persisted, I actually fully expected it to fail the same.<br></div><br>However to my surprise it did not fail, thus I assume something changed between yesterday morning this morning that allowed things to work.<br>
</div><br>for grins the packages I used yesterday were generated from:<br></div>engine commit: <span style="color:rgb(0,0,0);font-family:monospace;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);display:inline!important;float:none">ec2b57124edd5ae1fa41e986adadfefb9c5f1aa7</span></div>
otopi commit: <span style="color:rgb(0,0,0);font-family:monospace;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);display:inline!important;float:none">8e3e3c27cbcd4330143d571a3ce8caa36c6d20ef</span><br>
ovirt-host-deply commit: <span style="color:rgb(0,0,0);font-family:monospace;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);display:inline!important;float:none">35a5b5f5e6d6c240ef1d20904bc06498b0ed81e1</span><br>
<br></div>and the packages I generated this morning that worked were from:<br>engine commit: <span style="color:rgb(0,0,0);font-family:monospace;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);display:inline!important;float:none">28251d9acd1ecbe39b9b79e7e2e69342f57c9bd7</span><br>
otopi commit: <span style="color:rgb(0,0,0);font-family:monospace;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);display:inline!important;float:none">10892fbf3fe805cea416dfa3ba628f98ffa88b3e</span><br>
ovirt-host-deply commit: <span style="color:rgb(0,0,0);font-family:monospace;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);display:inline!important;float:none">e9c98c67dcdfca21091141bcb732727f50eb97d1<br>
<br></span></div><div><span style="color:rgb(0,0,0);font-family:monospace;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);display:inline!important;float:none">- DHC<br>
</span></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 30, 2013 at 3:27 PM, Doron Fediuck <span dir="ltr"><<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Big thanks DHC for your time and efforts.<br>
<br>
Since neither of us managed to reproduce the issue<br>
I suggest we drop it for now. If someone else will hit<br>
into it, we should try and get more info on a possible<br>
failure which will not complete the db upgrade.<br>
<br>
Thanks again,<br>
Doron<br>
<div class="im HOEnZb"><br>
----- Original Message -----<br>
| From: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>><br>
| To: "Doron Fediuck" <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| Cc: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>>, "users" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>>, "Dave Chen" <<a href="mailto:wei.d.chen@intel.com">wei.d.chen@intel.com</a>>, "Eli Mesika"<br>
| <<a href="mailto:emesika@redhat.com">emesika@redhat.com</a>><br>
</div><div class="HOEnZb"><div class="h5">| Sent: Tuesday, July 30, 2013 11:20:01 PM<br>
| Subject: Re: [Users] upgrade to latest master not working<br>
|<br>
| Ok, so something from yesterday morning to today changed. I again rebuilt<br>
| engine from the latest commit this AM. Accordingly I also built this<br>
| mornings latest commits of otopi and ovirt-host-deploy. I then attempted<br>
| upgrade and this time I did not note any major issues. The upgrade appears<br>
| to have succeeded, and engine starts and runs.<br>
| The only things that caused failures during upgrade were:<br>
| - If the previous version ovirt-engine-sdk is not removed manually and<br>
| replaced with ovirt-engine-sdk-python engine-upgrade fails due to package<br>
| dependency issues. (which then upsets ovirt-iso-uploader,<br>
| ovirt-image-uploader, ovirt-log-collector dependencies on ovirt-engine-sdk)<br>
| - If there is a leftover releasepreview directory engine-upgrade will not<br>
| run<br>
|<br>
| There were some error messages in the engine log griping about "Could not<br>
| parse option AutoRecoveryAllowedTypes value". I have attached the engine<br>
| log file from the first start of the engine after upgrade.<br>
|<br>
| The other minor thing to note is that the default ovirt.brand theme is not<br>
| working. The CSS styles are not being applied. Changes to<br>
| welcome_page.template and messages.properties were picked up however.<br>
|<br>
| - DHC<br>
|<br>
|<br>
|<br>
| On Tue, Jul 30, 2013 at 11:04 AM, Doron Fediuck <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>> wrote:<br>
|<br>
| ><br>
| ><br>
| > ----- Original Message -----<br>
| > | From: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>><br>
| > | To: "Doron Fediuck" <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| > | Cc: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>>, "users" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>>,<br>
| > "Dave Chen" <<a href="mailto:wei.d.chen@intel.com">wei.d.chen@intel.com</a>>, "Eli Mesika"<br>
| > | <<a href="mailto:emesika@redhat.com">emesika@redhat.com</a>><br>
| > | Sent: Tuesday, July 30, 2013 6:55:18 PM<br>
| > | Subject: Re: [Users] upgrade to latest master not working<br>
| > |<br>
| > | The only error I can find in the upgrade log is:<br>
| > | 2013-07-29 14:06:28 DEBUG otopi.plugins.otopi.packagers.yumpackager<br>
| > | yumpackager.verbose:84 Yum Done: ovirt-engine.noarch 0:3.3.0-20.fc18 - u<br>
| > | 2013-07-29 14:06:28 INFO otopi.plugins.otopi.packagers.yumpackager<br>
| > | <a href="http://yumpackager.info:88" target="_blank">yumpackager.info:88</a> Yum update: 10/18: ovirt-engine<br>
| > | 2013-07-29 14:06:35 DEBUG otopi.plugins.otopi.packagers.yumpackager<br>
| > | yumpackager.verbose:84 Yum Done: ovirt-engine.noarch 0:3.3.0-20.fc18 - u<br>
| > | 2013-07-29 14:06:35 DEBUG otopi.plugins.otopi.packagers.yumpackager<br>
| > | yumpackager.verbose:84 Yum Done: ovirt-engine-3.3.0-20.fc18.noarch<br>
| > | 2013-07-29 14:06:37 INFO otopi.plugins.otopi.packagers.yumpackager<br>
| > | <a href="http://yumpackager.info:88" target="_blank">yumpackager.info:88</a> Yum update: 11/18: ovirt-engine-dbscripts<br>
| > | 2013-07-29 14:06:38 DEBUG otopi.plugins.otopi.packagers.yumpackager<br>
| > | yumpackager.verbose:84 Yum Done:<br>
| > ovirt-engine-dbscripts-3.3.0-20.fc18.noarch<br>
| > | 2013-07-29 14:06:38 DEBUG otopi.plugins.otopi.packagers.yumpackager<br>
| > | yumpackager.verbose:84 Yum Done: ovirt-engine-backend<br>
| > | Traceback (most recent call last):<br>
| > | File "/usr/lib/python2.7/site-packages/yum/rpmtrans.py", line 464, in<br>
| > | callback<br>
| > | self._unInstStop( bytes, total, h )<br>
| > | File "/usr/lib/python2.7/site-packages/yum/rpmtrans.py", line 564, in<br>
| > | _unInstStop<br>
| > | self.total_actions)<br>
| > | File "/usr/lib/python2.7/site-packages/otopi/miniyum.py", line 204, in<br>
| > | event<br>
| > | package=<a href="http://package.name" target="_blank">package.name</a>,<br>
| > | AttributeError: 'str' object has no attribute 'name'<br>
| > | FATAL ERROR: python callback <bound method RPMTransaction.callback of<br>
| > | <yum.rpmtrans.RPMTransaction instance at 0x4587560>> failed, aborting!<br>
| > |<br>
| > | To test the database upgrade I will revert to a previous snapshot of the<br>
| > | debug VM, capture the dbase from the previous engine version, re-run<br>
| > | upgrade, dump the upgraded database, restore the previous, and then test<br>
| > a<br>
| > | manual run of the database upgrade script.<br>
| > |<br>
| > | Does this sound prudent?<br>
| > | - DHC<br>
| > |<br>
| ><br>
| > Yes. Although lots of work, it may show if there's a DB upgrade bug we<br>
| > missed, or indeed the general upgrade procedure has / had an issue.<br>
| ><br>
| > |<br>
| > | On Tue, Jul 30, 2013 at 5:54 AM, Doron Fediuck <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| > wrote:<br>
| > |<br>
| > | > Thanks.<br>
| > | > So this leads me further to the upgrade procedure<br>
| > | > which may have fail, thus not completing the DB upgrade.<br>
| > | > Can you please check the upgrade logs to see if something<br>
| > | > matches this flow?<br>
| > | ><br>
| > | > ----- Original Message -----<br>
| > | > | From: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>><br>
| > | > | To: "Doron Fediuck" <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| > | > | Cc: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>>, "users" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>>,<br>
| > | > "Dave Chen" <<a href="mailto:wei.d.chen@intel.com">wei.d.chen@intel.com</a>>, "Eli Mesika"<br>
| > | > | <<a href="mailto:emesika@redhat.com">emesika@redhat.com</a>><br>
| > | > | Sent: Tuesday, July 30, 2013 6:37:55 AM<br>
| > | > | Subject: Re: [Users] upgrade to latest master not working<br>
| > | > |<br>
| > | > | The engine version I am attempting to upgrade from was built from:<br>
| > | > | ---------------<br>
| > | > | commit: dbc3d31110ce372a1fa7e06f64c4a5c64544c679<br>
| > | > | tree: ee9ee41c3e855c26f491c77fda9622c05af3fc54<br>
| > | > | parent: 82644cae97f3c546ee631ae79c925c91e7bed836<br>
| > | > |<br>
| > | > | userportal,webadmin: Change remove message<br>
| > | > | Change-Id: Ia934e33d1a975e0235e1a1ffae0c8a4a7af66f10<br>
| > | > | Signed-off-by: Alexander Wels <<a href="mailto:awels@redhat.com">awels@redhat.com</a>><br>
| > | > | ---------------<br>
| > | > | Thus it is version 3.3 and not 3.2. The upgrade (built from this<br>
| > master<br>
| > | > | this AM) was attempted ovirt a fresh install of the packages built<br>
| > from<br>
| > | > the<br>
| > | > | above commit. I also confirmed the install was fully functional.<br>
| > | > |<br>
| > | > | - DHC<br>
| > | > |<br>
| > | > |<br>
| > | > |<br>
| > | > | On Mon, Jul 29, 2013 at 4:08 PM, Doron Fediuck <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| > | > wrote:<br>
| > | > |<br>
| > | > | ><br>
| > | > | ><br>
| > | > | > ----- Original Message -----<br>
| > | > | > | From: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>><br>
| > | > | > | To: "Doron Fediuck" <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| > | > | > | Cc: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>>, "users" <<br>
| > | > | > <a href="mailto:users@ovirt.org">users@ovirt.org</a>>, "Dave Chen" <<a href="mailto:wei.d.chen@intel.com">wei.d.chen@intel.com</a>>, "Eli<br>
| > | > | > | Mesika" <<a href="mailto:emesika@redhat.com">emesika@redhat.com</a>><br>
| > | > | > | Sent: Monday, July 29, 2013 11:51:35 PM<br>
| > | > | > | Subject: Re: [Users] upgrade to latest master not working<br>
| > | > | > |<br>
| > | > | > |<br>
| > | > | > |<br>
| > | > | > | ----- Original Message -----<br>
| > | > | > | > From: "Doron Fediuck" <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| > | > | > | > To: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>><br>
| > | > | > | > Cc: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>>, "users"<br>
| > | > | > | > <<a href="mailto:users@ovirt.org">users@ovirt.org</a>>, "Dave Chen" <<a href="mailto:wei.d.chen@intel.com">wei.d.chen@intel.com</a>>, "Eli<br>
| > | > | > | > Mesika" <<a href="mailto:emesika@redhat.com">emesika@redhat.com</a>><br>
| > | > | > | > Sent: Monday, July 29, 2013 11:43:46 PM<br>
| > | > | > | > Subject: Re: [Users] upgrade to latest master not working<br>
| > | > | > | ><br>
| > | > | > | ><br>
| > | > | > | ><br>
| > | > | > | > ----- Original Message -----<br>
| > | > | > | > | From: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>><br>
| > | > | > | > | To: "Doron Fediuck" <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| > | > | > | > | Cc: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>>, "users"<br>
| > | > | > | > | <<a href="mailto:users@ovirt.org">users@ovirt.org</a>>, "Dave Chen" <<a href="mailto:wei.d.chen@intel.com">wei.d.chen@intel.com</a>>, "Eli<br>
| > | > | > | > | Mesika" <<a href="mailto:emesika@redhat.com">emesika@redhat.com</a>><br>
| > | > | > | > | Sent: Monday, July 29, 2013 11:31:42 PM<br>
| > | > | > | > | Subject: Re: [Users] upgrade to latest master not working<br>
| > | > | > | > |<br>
| > | > | > | > |<br>
| > | > | > | > |<br>
| > | > | > | > | ----- Original Message -----<br>
| > | > | > | > | > From: "Doron Fediuck" <<a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>><br>
| > | > | > | > | > To: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>><br>
| > | > | > | > | > Cc: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>>, "users"<br>
| > | > | > | > | > <<a href="mailto:users@ovirt.org">users@ovirt.org</a>>, "Dave Chen" <<a href="mailto:wei.d.chen@intel.com">wei.d.chen@intel.com</a>>,<br>
| > "Eli<br>
| > | > | > | > | > Mesika" <<a href="mailto:emesika@redhat.com">emesika@redhat.com</a>><br>
| > | > | > | > | > Sent: Monday, July 29, 2013 11:27:41 PM<br>
| > | > | > | > | > Subject: Re: [Users] upgrade to latest master not working<br>
| > | > | > | > | ><br>
| > | > | > | > | ><br>
| > | > | > | > | ><br>
| > | > | > | > | > ----- Original Message -----<br>
| > | > | > | > | > | From: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>><br>
| > | > | > | > | > | To: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>><br>
| > | > | > | > | > | Cc: "users" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>>, "Doron Fediuck" <<br>
| > | > | > <a href="mailto:dfediuck@redhat.com">dfediuck@redhat.com</a>>,<br>
| > | > | > | > | > | "Dave<br>
| > | > | > | > | > | Chen" <<a href="mailto:wei.d.chen@intel.com">wei.d.chen@intel.com</a>>, "Eli<br>
| > | > | > | > | > | Mesika" <<a href="mailto:emesika@redhat.com">emesika@redhat.com</a>><br>
| > | > | > | > | > | Sent: Monday, July 29, 2013 10:47:01 PM<br>
| > | > | > | > | > | Subject: Re: [Users] upgrade to latest master not working<br>
| > | > | > | > | > |<br>
| > | > | > | > | > |<br>
| > | > | > | > | > |<br>
| > | > | > | > | > | ----- Original Message -----<br>
| > | > | > | > | > | > From: "Dead Horse" <<a href="mailto:deadhorseconsulting@gmail.com">deadhorseconsulting@gmail.com</a>><br>
| > | > | > | > | > | > To: "Alon Bar-Lev" <<a href="mailto:alonbl@redhat.com">alonbl@redhat.com</a>><br>
| > | > | > | > | > | > Cc: "users" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>><br>
| > | > | > | > | > | > Sent: Monday, July 29, 2013 10:41:34 PM<br>
| > | > | > | > | > | > Subject: Re: [Users] upgrade to latest master not<br>
| > working<br>
| > | > | > | > | > | ><br>
| > | > | > | > | > | > server.log attached<br>
| > | > | > | > | > |<br>
| > | > | > | > | > | Thanks!<br>
| > | > | > | > | > |<br>
| > | > | > | > | > | My guess is that I8ce3448a[1] broke master.<br>
| > | > | > | > | > |<br>
| > | > | > | > | > | [1] <a href="http://gerrit.ovirt.org/#/c/14605/" target="_blank">http://gerrit.ovirt.org/#/c/14605/</a><br>
| > | > | > | > | > |<br>
| > | > | > | > | ><br>
| > | > | > | > | > Works perfectly on my Gentoo with master.<br>
| > | > | > | > | ><br>
| > | > | > | > |<br>
| > | > | > | > | I doubt you are upgrading from 3.2 :)<br>
| > | > | > | > |<br>
| > | > | > | ><br>
| > | > | > | > Just did:<br>
| > | > | > | ><br>
| > | > | > | > $ git checkout origin/engine_3.2<br>
| > | > | > | > $ cd backend/manager/dbscripts/<br>
| > | > | > | > $ ./create_db.sh -d upgrade32-test -u engine<br>
| > | > | > | > ~/dev/ovirt/ovirt-engine/backend/manager/dbscripts<br>
| > | > | > | > ~/dev/ovirt/ovirt-engine/backend/manager/dbscripts<br>
| > | > | > | > Creating the database: upgrade32-test<br>
| > | > | > | > dropdb: database removal failed: ERROR: database<br>
| > "upgrade32-test"<br>
| > | > | > does not<br>
| > | > | > | > exist<br>
| > | > | > | > user name is: engine<br>
| > | > | > | > Creating tables...<br>
| > | > | > | > Creating functions...<br>
| > | > | > | > Creating common functions...<br>
| > | > | > | > Inserting data ...<br>
| > | > | > | > Inserting pre-defined roles ...<br>
| > | > | > | > Running upgrade scripts...<br>
| > | > | > | > upgrade script detected a change in Config, View or Stored<br>
| > | > Procedure...<br>
| > | > | > | > Running upgrade sql script<br>
| > upgrade/pre_upgrade/0000_config.sql ...<br>
| > | > | > | ><br>
| > | > | > | > ...<br>
| > | > | > | ><br>
| > | > | > | > Creating stored procedures from vm_templates_sp.sql ...<br>
| > | > | > | > Running upgrade sql script<br>
| > | > | > | ><br>
| > upgrade/post_upgrade/0010_add_object_column_white_list_table.sql<br>
| > | > ...<br>
| > | > | > | > Running upgrade sql script<br>
| > | > | > | > upgrade/post_upgrade/0020_create_materialized_views.sql ...<br>
| > | > | > | > Refreshing materialized views...<br>
| > | > | > | > ~/dev/ovirt/ovirt-engine/backend/manager/dbscripts<br>
| > | > | > | ><br>
| > | > | > | > $ git checkout master<br>
| > | > | > | > $ cd ../../../packaging/dbscripts/<br>
| > | > | > | > $ ./upgrade.sh -d upgrade32-test -u engine<br>
| > | > | > | > upgrade script detected a change in Config, View or Stored<br>
| > | > Procedure...<br>
| > | > | > | > Running upgrade sql script upgrade/pre_upgrade/0000_config.sql<br>
| > ...<br>
| > | > | > | ><br>
| > | > | > | > ...<br>
| > | > | > | ><br>
| > | > | > | > Creating stored procedures from vms_sp.sql ...<br>
| > | > | > | > Running upgrade sql script<br>
| > | > | > | ><br>
| > upgrade/post_upgrade/0010_add_object_column_white_list_table.sql<br>
| > | > ...<br>
| > | > | > | > Running upgrade sql script<br>
| > | > | > | > upgrade/post_upgrade/0020_create_materialized_views.sql ...<br>
| > | > | > | > Refreshing materialized views...<br>
| > | > | > | > Done.<br>
| > | > | > | ><br>
| > | > | > | ><br>
| > | > | > | > Seems to work well. Am I missing something?<br>
| > | > | > | ><br>
| > | > | > |<br>
| > | > | > | Start jboss?<br>
| > | > | > |<br>
| > | > | > | Please take over this issue, you have relevant logs, and a<br>
| > | > cooperative<br>
| > | > | > user.<br>
| > | > | > |<br>
| > | > | > | Thanks,<br>
| > | > | > | Alon<br>
| > | > | > |<br>
| > | > | ><br>
| > | > | > Since jboss is up and running very well after upgrade I do not see<br>
| > a<br>
| > | > | > problem.<br>
| > | > | > Is it possible that the install / upgrade process didn't complete<br>
| > due<br>
| > | > to<br>
| > | > | > other<br>
| > | > | > errors ending up with this state?<br>
| > | > | ><br>
| > | > | > In the meantime you can manually create the missing function to get<br>
| > | > your<br>
| > | > | > setup running. The only problem is that if my assumption is<br>
| > correct on<br>
| > | > | > upgrade not finishing, you will hit the next db change as an<br>
| > error. So<br>
| > | > | > I'd try and look at the install/upgrade log files to see if db<br>
| > upgrade<br>
| > | > | > failed at some point.<br>
| > | > | ><br>
| > | > |<br>
| > | ><br>
| > |<br>
| ><br>
|<br>
</div></div></blockquote></div><br></div>