<div dir="ltr">I'm seeing a failure in 4.1, which I've never seen in the past:<div>2017-01-26 02:21:00,124 ERROR (mailbox-spm) [storage.MailBox.SpmMailMonitor] SPM_MailMonitor: mailbox 2 checksum failed, not clearing mailbox, clearing newMail. (storage_mailbox:618)<br></div><div><br></div><div>(from host1).</div><div><br></div><div>There are also some new warnings around monitoring, such as:</div><div>2017-01-26 02:20:58,035 WARN (libvirt/events) [virt.vm] (vmId='85e8df3c-8bdf-470f-8c28-baa521d663de') unknown eventid 8 args ('/rhev/data-center/27d461cd-03b2-4b2b-a1f8-157c27bb710e/2e10aa41-24f1-4c83-8e29-15f772a2d04b/images/304c3f03-61a6-4b79-a748-174a4cd24f84/3f96ca05-e8f3-41f2-8b9c-c7d170b24f92', 3, 0, 8) (clientIF:540)<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 25, 2017 at 2:17 PM, Gil Shinar <span dir="ltr"><<a href="mailto:gshinar@redhat.com" target="_blank">gshinar@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Link to job: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/4951/" target="_blank">http://jenkins.ovirt.org/<wbr>job/test-repo_ovirt_<wbr>experimental_master/4951/</a><div><br></div><div>Sorry for the mistake</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 25, 2017 at 1:43 PM, Gil Shinar <span dir="ltr"><<a href="mailto:gshinar@redhat.com" target="_blank">gshinar@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>The test in $subject has failed. Below please find exception I have found in engine.log.</div><div><pre style="color:rgb(0,0,0)">{"jsonrpc": "2.0", "id": "4b8eb810-c52d-4c92-a792-e554f<wbr>87c9493", "error": {"message": "Cannot deactivate Logical Volume: ('General Storage Exception: (\"5 [] [\\' WARNING: Not using lvmetad because config setting use_lvmetad=0.\\', \\' WARNING: To avoid corruption, rescan devices to make changes visible (pvscan --cache).\\', \\' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/778cbc5b-a9df-46d7-<wbr>bc80-1a66f7d3e2b5 in use.\\', \\' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/ab4e8962-6196-485e-<wbr>be2a-d5791a38eaeb in use.\\']\\\\nf9dce023-0282-418<wbr>5-9ad9-fe71c3975106/[\\'778cbc<wbr>5b-a9df-46d7-bc80-1a66f7d3e2b5<wbr>\\', \\'ab4e8962-6196-485e-be2a-d57<wbr>91a38eaeb\\']\",)',)", "code": 552}}�<br></pre><pre style="color:rgb(0,0,0)">2017-01-25 05:05:49,037-05 DEBUG [org.ovirt.vdsm.jsonrpc.client<wbr>.internal.ResponseWorker] (ResponseWorker) [] Message received: {"jsonrpc": "2.0", "id": "4b8eb810-c52d-4c92-a792-e554f<wbr>87c9493", "error": {"message": "Cannot deactivate Logical Volume: ('General Storage Exception: (\"5 [] [\\' WARNING: Not using lvmetad because config setting use_lvmetad=0.\\', \\' WARNING: To avoid corruption, rescan devices to make changes visible (pvscan --cache).\\', \\' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/778cbc5b-a9df-46d7-<wbr>bc80-1a66f7d3e2b5 in use.\\', \\' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/ab4e8962-6196-485e-<wbr>be2a-d5791a38eaeb in use.\\']\\\\nf9dce023-0282-418<wbr>5-9ad9-fe71c3975106/[\\'778cbc<wbr>5b-a9df-46d7-bc80-1a66f7d3e2b5<wbr>\\', \\'ab4e8962-6196-485e-be2a-d57<wbr>91a38eaeb\\']\",)',)", "code": 552}}
2017-01-25 05:05:49,047-05 ERROR [org.ovirt.engine.core.dal.dbb<wbr>roker.auditloghandling.AuditLo<wbr>gDirector] (DefaultQuartzScheduler5) [59ab00f1] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,<wbr>802), Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM lago-basic-suite-master-host0 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' WARNING: Not using lvmetad because config setting use_lvmetad=0.\', \' WARNING: To avoid corruption, rescan devices to make changes visible (pvscan --cache).\', \' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/778cbc5b-a9df-46d7-<wbr>bc80-1a66f7d3e2b5 in use.\', \' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/ab4e8962-6196-485e-<wbr>be2a-d5791a38eaeb in use.\']\\nf9dce023-0282-4185-9<wbr>ad9-fe71c3975106/[\'778cbc5b-a<wbr>9df-46d7-bc80-1a66f7d3e2b5\', \'ab4e8962-6196-485e-be2a-d579<wbr>1a38eaeb\']",)',)
2017-01-25 05:05:49,047-05 ERROR [org.ovirt.engine.core.vdsbrok<wbr>er.vdsbroker.TeardownImageVDSC<wbr>ommand] (DefaultQuartzScheduler5) [59ab00f1] Command 'TeardownImageVDSCommand(HostN<wbr>ame = lago-basic-suite-master-host0, ImageActionsVDSCommandParamete<wbr>rs:{runAsync='true', hostId='60e52527-b637-445c-b40<wbr>8-0275d347e76a'})' execution failed: VDSGenericException: VDSErrorException: Failed in vdscommand to TeardownImageVDS, error = Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' WARNING: Not using lvmetad because config setting use_lvmetad=0.\', \' WARNING: To avoid corruption, rescan devices to make changes visible (pvscan --cache).\', \' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/778cbc5b-a9df-46d7-<wbr>bc80-1a66f7d3e2b5 in use.\', \' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/ab4e8962-6196-485e-<wbr>be2a-d5791a38eaeb in use.\']\\nf9dce023-0282-4185-9<wbr>ad9-fe71c3975106/[\'778cbc5b-a<wbr>9df-46d7-bc80-1a66f7d3e2b5\', \'ab4e8962-6196-485e-be2a-d579<wbr>1a38eaeb\']",)',)
2017-01-25 05:05:49,047-05 DEBUG [org.ovirt.engine.core.vdsbrok<wbr>er.vdsbroker.TeardownImageVDSC<wbr>ommand] (DefaultQuartzScheduler5) [59ab00f1] Exception: org.ovirt.engine.core.vdsbroke<wbr>r.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed in vdscommand to TeardownImageVDS, error = Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' WARNING: Not using lvmetad because config setting use_lvmetad=0.\', \' WARNING: To avoid corruption, rescan devices to make changes visible (pvscan --cache).\', \' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/778cbc5b-a9df-46d7-<wbr>bc80-1a66f7d3e2b5 in use.\', \' Logical volume f9dce023-0282-4185-9ad9-fe71c3<wbr>975106/ab4e8962-6196-485e-<wbr>be2a-d5791a38eaeb in use.\']\\nf9dce023-0282-4185-9<wbr>ad9-fe71c3975106/[\'778cbc5b-a<wbr>9df-46d7-bc80-1a66f7d3e2b5\', \'ab4e8962-6196-485e-be2a-d579<wbr>1a38eaeb\']",)',)
at org.ovirt.engine.core.vdsbroke<wbr>r.vdsbroker.BrokerCommandBase.<wbr>proceedProxyReturnValue(Broker<wbr>CommandBase.java:182) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroke<wbr>r.vdsbroker.ImageActionsVDSCom<wbr>mandBase.executeVdsBrokerComma<wbr>nd(ImageActionsVDSCommandBase.<wbr>java:20) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroke<wbr>r.vdsbroker.VdsBrokerCommand.e<wbr>xecuteVDSCommand(VdsBrokerComm<wbr>and.java:111) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroke<wbr>r.VDSCommandBase.executeComman<wbr>d(VDSCommandBase.java:73) [vdsbroker.jar:]
at org.ovirt.engine.core.dal.VdcC<wbr>ommandBase.execute(VdcCommandB<wbr>ase.java:33) [dal.jar:]
at org.ovirt.engine.core.vdsbroke<wbr>r.vdsbroker.DefaultVdsCommandE<wbr>xecutor.execute(DefaultVdsComm<wbr>andExecutor.java:14) [vdsbroker.jar:]
at org.ovirt.engine.core.vdsbroke<wbr>r.ResourceManager.runVdsComman<wbr>d(ResourceManager.java:407) [vdsbroker.jar:]
at org.ovirt.engine.core.bll.VDSB<wbr>rokerFrontendImpl.runVdsComman<wbr>d(VDSBrokerFrontendImpl.java:<wbr>33) [bll.jar:]
at org.ovirt.engine.core.bll.stor<wbr>age.disk.image.ImagesHandler.<wbr>teardownImage(ImagesHandler.<wbr>java:1007) [bll.jar:]
at org.ovirt.engine.core.bll.stor<wbr>age.disk.image.ImagesHandler.g<wbr>etQemuImageInfoFromVdsm(Images<wbr>Handler.java:856) [bll.jar:]
at org.ovirt.engine.core.bll.stor<wbr>age.disk.image.BaseImagesComma<wbr>nd.endSuccessfully(BaseImagesC<wbr>ommand.java:367) [bll.jar:]
at org.ovirt.engine.core.bll.Comm<wbr>andBase.internalEndSuccessfull<wbr>y(CommandBase.java:736) [bll.jar:]
at org.ovirt.engine.core.bll.Comm<wbr>andBase.endActionInTransaction<wbr>Scope(CommandBase.java:694) [bll.jar:]
at org.ovirt.engine.core.bll.Comm<wbr>andBase.runInTransaction(Comma<wbr>ndBase.java:2057) [bll.jar:]
at <a href="http://org.ovirt.engine.core.utils.tr">org.ovirt.engine.core.utils.tr</a><wbr>ansaction.TransactionSupport.e<wbr>xecuteInSuppressed(Transaction<wbr>Support.java:164) [utils.jar:]
at <a href="http://org.ovirt.engine.core.utils.tr">org.ovirt.engine.core.utils.tr</a><wbr>ansaction.TransactionSupport.e<wbr>xecuteInScope(TransactionSuppo<wbr>rt.java:103) [utils.jar:]
at org.ovirt.engine.core.bll.Comm<wbr>andBase.endAction(CommandBase.<wbr>java:559) [bll.jar:]
at org.ovirt.engine.core.bll.Back<wbr>end.endAction(Backend.java:<wbr>536) [bll.jar:]</pre><pre style="color:rgb(0,0,0)"><br></pre>Link to suspected patches: <a href="https://gerrit.ovirt.org/#/c/71132/" target="_blank">https://gerrit.ovirt.org/#/c/7<wbr>1132/</a><br>Link to Job: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/4950/" target="_blank">http://jenkins.ovirt.org/job/t<wbr>est-repo_ovirt_experimental_ma<wbr>ster/4950/</a><br>Link to all logs: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/4951/artifact/exported-artifacts/basic_suite_master.sh-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/" target="_blank">http://jenkins.ovirt.org/job/t<wbr>est-repo_ovirt_experimental_ma<wbr>ster/4951/artifact/exported-ar<wbr>tifacts/basic_suite_master.sh-<wbr>el7/test_logs/basic-suite-mast<wbr>er/post-004_basic_sanity.py/</a></div><div><br></div><div>Thanks</div><span class="m_1209065967114490674HOEnZb"><font color="#888888"><div>Gil</div></font></span></div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/devel</a><br></blockquote></div><br></div>