[ovirt-users] Re: General failure

Hi Marcelo, Do you mean copying the whole disk block to a different device and attaching it to a new VM? Anything will be appreciated, as currently we're facing a distressing situation, so if you can describe what you mean I'd be grateful. Thanks. El 2018-06-18 16:19, Marcelo Leandro escribió: > Hello, > Do you can copy diskbase to a new vm. > > If you want I can describe the step. > > Em seg, 18 de jun de 2018 11:49, <nicolas(a)devels.es&gt; escreveu: > > Indeed, when the problem started I think the SPM was the host I >> added as >> VDSM log in the first e-mail. Currently it is the one I sent in the >> >> second mail. >> >> FWIW, if it helps to debug more fluently, we can provide VPN access >> to >> our infrastructure so you can access and see whateve you need (all >> hosts, DB, etc...). >> >> Right now the machines that keep running work, but once shut down >> they >> start showing the problem below... >> >> Thank you >> >> El 2018-06-18 15:20, Benny Zlotnik escribió: >> >>> I'm having trouble following the errors, I think the SPM changed >>> >> or >> >>> the vdsm log from the right host might be missing. >>> >>> However, I believe what started the problems is this transaction >>> timeout: >>> >>> 2018-06-15 14:20:51,378+01 ERROR >>> [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] >>> (org.ovirt.thread.pool-6-thread-29) >>> [1db468cb-85fd-4189-b356-d31781461504] [within thread]: endAction >>> >> for >> >>> action type RemoveSnapshotSingleDisk threw an exception.: >>> org.springframework.jdbc.CannotGetJdbcConnectionException: Could >>> >> not >> >>> get JDBC Connection; nested exception is java.sql.SQLException: >>> javax.resource.ResourceException: IJ000460: Error checking for a >>> transaction >>> at >>> >>> >> org.springframework.jdbc.datasource.DataSourceUtils.getConne > ction(DataSourceUtils.java:80) > >> [spring-jdbc.jar:4.2.4.RELEASE] >>> at >>> >>> >> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTempl > ate.java:615) > >> [spring-jdbc.jar:4.2.4.RELEASE] >>> at >>> >>> >> org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:680) > >> [spring-jdbc.jar:4.2.4.RELEASE] >>> at >>> >>> >> org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:712) > >> [spring-jdbc.jar:4.2.4.RELEASE] >>> at >>> >>> >> org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:762) > >> [spring-jdbc.jar:4.2.4.RELEASE] >>> at >>> >>> >> org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$P > ostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDi > alect.java:152) > >> [dal.jar:] >>> >>> This looks like a bug >>> >>> Regardless, I am not sure restoring a backup would help since you >>> probably have orphaned images on the storage which need to be >>> >> removed >> >>> >>> Adding Ala >>> >>> On Mon, Jun 18, 2018 at 4:19 PM, <nicolas(a)devels.es&gt; wrote: >>> >>> Hi Benny, >>>> >>>> Please find the SPM logs at [1]. >>>> >>>> Thank you >>>> >>>> [1]: >>>> >>>> >>> >> https://wetransfer.com/downloads/62bf649462aabbc2ef21824682b > 0a08320180618131825/036b7782f58d337baf909a7220d8455320180618131825/5550ee > >> [1] >> >>> [1] >>>> >>>> El 2018-06-18 13:19, Benny Zlotnik escribió: >>>> Can you send the SPM logs as well? >>>> >>>> On Mon, Jun 18, 2018 at 1:13 PM, <nicolas(a)devels.es&gt; wrote: >>>> >>>> Hi Benny, >>>> >>>> Please find the logs at [1]. >>>> >>>> Thank you. >>>> >>>> [1]: >>>> >>>> >>>> >>> >> https://wetransfer.com/downloads/12208fb4a6a5df3114bbbc10af1 > 94c8820180618101223/647c066b7b91096570def304da86dbca20180618101223/583d3d > >> [2] >> >>> [2] >>>> [1] >>>> >>>> El 2018-06-18 09:28, Benny Zlotnik escribió: >>>> >>>> Can you provide full engine and vdsm logs? >>>> >>>> On Mon, Jun 18, 2018 at 11:20 AM, <nicolas(a)devels.es&gt; wrote: >>>> >>>> Hi, >>>> >>>> We're running oVirt 4.1.9 (we cannot upgrade at this time) and >>>> we're having a major problem in our infrastructure. On friday, a >>>> snapshots were automatically created on more than 200 VMs and as >>>> this was just a test task, all of them were deleted at the same >>>> time, which seems to have corrupted several VMs. >>>> >>>> When trying to delete a snapshot on some of the VMs, a "General >>>> error" is thrown with a NullPointerException in the engine log >>>> (attached). >>>> >>>> But the worst part is that when some of these machines is >>>> >>> powered >> >>> off and then powered on, the VMs are corrupt... >>>> >>>> VM myvm is down with error. Exit message: Bad volume >>>> >>> specification >> >>> {u'index': 0, u'domainID': >>>> >>> u'110ea376-d789-40a1-b9f6-6b40c31afe01', >> >>> 'reqsize': '0', u'format': u'cow', u'bootOrder': u'1', >>>> >>> u'address': >> >>> {u'function': u'0x0', u'bus': u'0x00', u'domain': u'0x0000', >>>> u'type': u'pci', u'slot': u'0x06'}, u'volumeID': >>>> u'1fd0f9aa-6505-45d2-a17e-859bd5dd4290', 'apparentsize': >>>> '23622320128', u'imageID': >>>> >>> u'65519220-68e1-462a-99b3-f0763c78eae2', >> >>> u'discard': False, u'specParams': {}, u'readonly': u'false', >>>> u'iface': u'virtio', u'optional': u'false', u'deviceId': >>>> u'65519220-68e1-462a-99b3-f0763c78eae2', 'truesize': >>>> >>> '23622320128', >> >>> u'poolID': u'75bf8f48-970f-42bc-8596-f8ab6efb2b63', u'device': >>>> u'disk', u'shared': u'false', u'propagateErrors': u'off', >>>> >>> u'type': >> >>> u'disk'}. >>>> >>>> We're really frustrated by now and don't know how to procceed... >>>> >>> We >> >>> have a DB backup (with engine-backup) from thursday which would >>>> have >>>> a "sane" DB definition without all the snapshots, as they were >>>> >>> all >> >>> created on friday. Would it be safe to restore this backup? >>>> >>>> Any help is really appreciated... >>>> >>>> Thanks. >>>> _______________________________________________ >>>> Users mailing list -- users(a)ovirt.org >>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> >>> [3] [3] >> >>> [2] >>>> [1] >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ [4] >>>> >>> [4] [3] >> >>> [2] >>>> List Archives: >>>> >>>> >>>> >>> >> https://lists.ovirt.org/archives/list/users@ovirt.org/messag > e/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/ > >> [5] >> >>> [5] >>>> [4] >>>> [3] >>>> >>>> Links: >>>> ------ >>>> [1] https://www.ovirt.org/site/privacy-policy/ [3] [3] [2] >>>> [2] https://www.ovirt.org/community/about/community-guidelines/ >>>> >>> [4] [4] >> >>> [3] >>>> [3] >>>> >>>> >>>> >>> >> https://lists.ovirt.org/archives/list/users@ovirt.org/messag > e/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/ > >> [5] >> >>> [5] >>>> [4] >>>> >>>> Links: >>>> ------ >>>> [1] >>>> >>>> >>> >> https://wetransfer.com/downloads/12208fb4a6a5df3114bbbc10af1 > 94c8820180618101223/647c066b7b91096570def304da86dbca20180618101223/583d3d > >> [2] >> >>> [2] >>>> [2] https://www.ovirt.org/site/privacy-policy/ [3] [3] >>>> [3] https://www.ovirt.org/community/about/community-guidelines/ >>>> >>> [4] [4] >> >>> [4] >>>> >>>> >>> >> https://lists.ovirt.org/archives/list/users@ovirt.org/messag > e/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/ > >> [5] >> >>> [5] >>>> >>> >>> >>> >>> Links: >>> ------ >>> [1] >>> >>> >> https://wetransfer.com/downloads/62bf649462aabbc2ef21824682b > 0a08320180618131825/036b7782f58d337baf909a7220d8455320180618131825/5550ee > >> [1] >> >>> [2] >>> >>> >> https://wetransfer.com/downloads/12208fb4a6a5df3114bbbc10af1 > 94c8820180618101223/647c066b7b91096570def304da86dbca20180618101223/583d3d > >> [2] >> >>> [3] https://www.ovirt.org/site/privacy-policy/ [3] >>> [4] https://www.ovirt.org/community/about/community-guidelines/ >>> >> [4] >> >>> [5] >>> >>> >> https://lists.ovirt.org/archives/list/users@ovirt.org/messag > e/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/ > >> [5] >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ [3] >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ [4] >> List Archives: >> >> https://lists.ovirt.org/archives/list/users@ovirt.org/messag > e/GS46O7WRFAWPLEBHDCYBGVPSN4LBOJ3Y/ > >> [6] >> > > > Links: > ------ > [1] > https://wetransfer.com/downloads/62bf649462aabbc2ef21824682b > 0a08320180618131825/036b7782f58d337baf909a7220d8455320180618131825/5550ee > [2] > https://wetransfer.com/downloads/12208fb4a6a5df3114bbbc10af1 > 94c8820180618101223/647c066b7b91096570def304da86dbca20180618101223/583d3d > [3] https://www.ovirt.org/site/privacy-policy/ > [4] https://www.ovirt.org/community/about/community-guidelines/ > [5] > https://lists.ovirt.org/archives/list/users@ovirt.org/messag > e/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/ > [6] > https://lists.ovirt.org/archives/list/users@ovirt.org/messag > e/GS46O7WRFAWPLEBHDCYBGVPSN4LBOJ3Y/ >