Can you add the server.log?

On Mon, Jun 18, 2018 at 5:46 PM, <nicolas@devels.es> wrote:
Indeed, when the problem started I think the SPM was the host I added as VDSM log in the first e-mail. Currently it is the one I sent in the second mail.

FWIW, if it helps to debug more fluently, we can provide VPN access to our infrastructure so you can access and see whateve you need (all hosts, DB, etc...).

Right now the machines that keep running work, but once shut down they start showing the problem below...

Thank you


El 2018-06-18 15:20, Benny Zlotnik escribió:
I'm having trouble following the errors, I think the SPM changed or
the vdsm log from the right host might be missing.

However, I believe what started the problems is this transaction
timeout:

2018-06-15 14:20:51,378+01 ERROR
[org.ovirt.engine.core.bll.tasks.CommandAsyncTask]
(org.ovirt.thread.pool-6-thread-29)
[1db468cb-85fd-4189-b356-d31781461504] [within thread]: endAction for
action type RemoveSnapshotSingleDisk threw an exception.:
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not
get JDBC Connection; nested exception is java.sql.SQLException:
javax.resource.ResourceException: IJ000460: Error checking for a
transaction
 at
org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
[spring-jdbc.jar:4.2.4.RELEASE]
 at
org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:615)
[spring-jdbc.jar:4.2.4.RELEASE]
 at
org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:680)
[spring-jdbc.jar:4.2.4.RELEASE]
 at
org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:712)
[spring-jdbc.jar:4.2.4.RELEASE]
 at
org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:762)
[spring-jdbc.jar:4.2.4.RELEASE]
 at
org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:152)
[dal.jar:]

This looks like a bug

Regardless, I am not sure restoring a backup would help since you
probably have orphaned images on the storage which need to be removed

Adding Ala

On Mon, Jun 18, 2018 at 4:19 PM, <nicolas@devels.es> wrote:

Hi Benny,

Please find the SPM logs at [1].

Thank you

  [1]:

https://wetransfer.com/downloads/62bf649462aabbc2ef21824682b0a08320180618131825/036b7782f58d337baf909a7220d8455320180618131825/5550ee
[1]

El 2018-06-18 13:19, Benny Zlotnik escribió:
Can you send the SPM logs as well?

On Mon, Jun 18, 2018 at 1:13 PM, <nicolas@devels.es> wrote:

Hi Benny,

Please find the logs at [1].

Thank you.

  [1]:


https://wetransfer.com/downloads/12208fb4a6a5df3114bbbc10af194c8820180618101223/647c066b7b91096570def304da86dbca20180618101223/583d3d
[2]

[1]

El 2018-06-18 09:28, Benny Zlotnik escribió:

Can you provide full engine and vdsm logs?

On Mon, Jun 18, 2018 at 11:20 AM, <nicolas@devels.es> wrote:

Hi,

We're running oVirt 4.1.9 (we cannot upgrade at this time) and
we're having a major problem in our infrastructure. On friday, a
snapshots were automatically created on more than 200 VMs and as
this was just a test task, all of them were deleted at the same
time, which seems to have corrupted several VMs.

When trying to delete a snapshot on some of the VMs, a "General
error" is thrown with a NullPointerException in the engine log
(attached).

But the worst part is that when some of these machines is powered
off and then powered on, the VMs are corrupt...

VM myvm is down with error. Exit message: Bad volume specification
{u'index': 0, u'domainID': u'110ea376-d789-40a1-b9f6-6b40c31afe01',
'reqsize': '0', u'format': u'cow', u'bootOrder': u'1', u'address':
{u'function': u'0x0', u'bus': u'0x00', u'domain': u'0x0000',
u'type': u'pci', u'slot': u'0x06'}, u'volumeID':
u'1fd0f9aa-6505-45d2-a17e-859bd5dd4290', 'apparentsize':
'23622320128', u'imageID': u'65519220-68e1-462a-99b3-f0763c78eae2',
u'discard': False, u'specParams': {}, u'readonly': u'false',
u'iface': u'virtio', u'optional': u'false', u'deviceId':
u'65519220-68e1-462a-99b3-f0763c78eae2', 'truesize': '23622320128',
u'poolID': u'75bf8f48-970f-42bc-8596-f8ab6efb2b63', u'device':
u'disk', u'shared': u'false', u'propagateErrors': u'off', u'type':
u'disk'}.

We're really frustrated by now and don't know how to procceed... We
have a DB backup (with engine-backup) from thursday which would
have
a "sane" DB definition without all the snapshots, as they were all
created on friday. Would it be safe to restore this backup?

Any help is really appreciated...

Thanks.
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/ [3]
[2]
[1]
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/ [4] [3]
[2]
List Archives:


https://lists.ovirt.org/archives/list/users@ovirt.org/message/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/
[5]
[4]
[3]

Links:
------
[1] https://www.ovirt.org/site/privacy-policy/ [3] [2]
[2] https://www.ovirt.org/community/about/community-guidelines/ [4]
[3]
[3]


https://lists.ovirt.org/archives/list/users@ovirt.org/message/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/
[5]
[4]

Links:
------
[1]

https://wetransfer.com/downloads/12208fb4a6a5df3114bbbc10af194c8820180618101223/647c066b7b91096570def304da86dbca20180618101223/583d3d
[2]
[2] https://www.ovirt.org/site/privacy-policy/ [3]
[3] https://www.ovirt.org/community/about/community-guidelines/ [4]
[4]

https://lists.ovirt.org/archives/list/users@ovirt.org/message/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/
[5]



Links:
------
[1]
https://wetransfer.com/downloads/62bf649462aabbc2ef21824682b0a08320180618131825/036b7782f58d337baf909a7220d8455320180618131825/5550ee
[2]
https://wetransfer.com/downloads/12208fb4a6a5df3114bbbc10af194c8820180618101223/647c066b7b91096570def304da86dbca20180618101223/583d3d
[3] https://www.ovirt.org/site/privacy-policy/
[4] https://www.ovirt.org/community/about/community-guidelines/
[5]
https://lists.ovirt.org/archives/list/users@ovirt.org/message/P5OOGBL3BRZIQ2I46FYELBUIIWT5QK4C/