This is a multi-part message in MIME format. ------=_NextPartTM-000-97ba05ae-423e-4aa1-b523-bb7ca0a2f1ac Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi there,=0A= =0A= we got a nasty situation yesterday in our OVirt 3.5.6 environment. =0A= We ran a LSM that failed during the cleanup operation. To be precise =0A= when the process deleted an image on the source NFS storage. =0A= =0A= Engine log gives:=0A= =0A= 2016-01-11 20:49:45,120 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.De= leteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-14) [77277f0] STA= RT, DeleteImageGroupVDSCommand( storagePoolId =3D 94ed7a19-fade-4bd6-83f2-2= cbb2f730b95, ignoreFailoverLimit =3D false, storageDomainId =3D 272ec473-60= 41-42ee-bd1a-732789dd18d4, imageGroupId =3D aed132ef-703a-44d0-b875-db8c0d2= c1a92, postZeros =3D false, forceDelete =3D false), log id: b52d59c=0A= ...=0A= 2016-01-11 20:50:45,206 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.De= leteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-14) [77277f0] Fai= led in DeleteImageGroupVDS method=0A= =0A= VDSM (SPM) log gives:=0A= =0A= Thread-97::DEBUG::2016-01-11 20:49:45,737::fileSD::384::Storage.StorageDoma= in::(deleteImage) Removing file: /rhev/data-center/mnt/1.2.3.4:_var_nas2_OV= irtIB/272ec473-6041-42ee-bd1a-732789dd18d4/images/_remojzBd1r/0d623afb-291e= -4f4c-acba-caecb125c4ed=0A= ...=0A= Thread-97::ERROR::2016-01-11 20:50:45,737::task::866::Storage.TaskManager.T= ask::(_setError) Task=3D`cd477878-47b4-44b1-85a3-b5da19543a5e`::Unexpected = error=0A= Traceback (most recent call last):=0A= File "/usr/share/vdsm/storage/task.py", line 873, in _run=0A= return fn(*args, **kargs)=0A= File "/usr/share/vdsm/logUtils.py", line 45, in wrapper=0A= res =3D f(*args, **kwargs)=0A= File "/usr/share/vdsm/storage/hsm.py", line 1549, in deleteImage=0A= pool.deleteImage(dom, imgUUID, volsByImg)=0A= File "/usr/share/vdsm/storage/securable.py", line 77, in wrapper=0A= return method(self, *args, **kwargs)=0A= File "/usr/share/vdsm/storage/sp.py", line 1884, in deleteImage=0A= domain.deleteImage(domain.sdUUID, imgUUID, volsByImg)=0A= File "/usr/share/vdsm/storage/fileSD.py", line 385, in deleteImage=0A= self.oop.os.remove(volPath)=0A= File "/usr/share/vdsm/storage/outOfProcess.py", line 245, in remove=0A= self._iop.unlink(path)=0A= File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 455, = in unlink=0A= return self._sendCommand("unlink", {"path": path}, self.timeout)=0A= File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 385, = in _sendCommand=0A= raise Timeout(os.strerror(errno.ETIMEDOUT))=0A= Timeout: Connection timed out=0A= =0A= Reading the docs I got the idea that vdsm default 60 second timeout=0A= for IO operations might be changed within /etc/vdsm/vdsm.conf=0A= =0A= [irs]=0A= process_pool_timeout =3D 180=0A= =0A= Can anyone confirm that this will solve the problem?=0A= =0A= Markus=0A= =0A= =0A= =0A= =0A= =0A= ------=_NextPartTM-000-97ba05ae-423e-4aa1-b523-bb7ca0a2f1ac Content-Type: text/plain; name="InterScan_Disclaimer.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="InterScan_Disclaimer.txt" **************************************************************************** Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet. Über das Internet versandte E-Mails können unter fremden Namen erstellt oder manipuliert werden. Deshalb ist diese als E-Mail verschickte Nachricht keine rechtsverbindliche Willenserklärung. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln Vorstand: Kadir Akin Dr. Michael Höhnerbach Vorsitzender des Aufsichtsrates: Hans Kristian Langva Registergericht: Amtsgericht Köln Registernummer: HRB 52 497 This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. e-mails sent over the internet may have been written under a wrong name or been manipulated. That is why this message sent as an e-mail is not a legally binding declaration of intention. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln executive board: Kadir Akin Dr. Michael Höhnerbach President of the supervisory board: Hans Kristian Langva Registry office: district court Cologne Register number: HRB 52 497 **************************************************************************** ------=_NextPartTM-000-97ba05ae-423e-4aa1-b523-bb7ca0a2f1ac--