Re: [ovirt-devel] [ovirt-users] spm changes hourly due to unexpected error of getAllTasksStatuses

This is a multi-part message in MIME format. ------=_001_NextPart870643344860_=---- Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 VGhhbmsgeW91IGZvciB5b3VyIHJlcGx5Lg0KDQpUaGUgcmVzdWx0IG9mIGJvdGggaG9zdHMgaXMg emhfQ04udXRmOA0KQnV0IGkgdGhpbmsgbWF5IGJlIGl0J3Mgbm90IHRoZSBwcm9ibGVtIG9mIGxv Y2FsZSwgDQpiZWNhdXNlIHdoZW4gaSBydW4gJ3Zkc0NsaWVudCAtcyAwIGdldEFsbFRhc2tzU3Rh dHVzZXMnIG1hbnVhbGx5IG9uIHRoZSBTUE0gaG9zdCwgdGhlcmUgaXMgbm8gcHJvYmxlbShubyBl cnJvcnMgaW4gdmRzbS5sb2cpDQoNCg0KDQpsaWtlLm1hQGNzMmMuY29tLmNuDQogDQpGcm9tOiBF bGkgTWVzaWthDQpEYXRlOiAyMDE1LTA2LTIzIDE2OjQ0DQpUbzogbGlrZSBtYQ0KQ0M6IGRldmVs OyB1c2Vycw0KU3ViamVjdDogUmU6IFtvdmlydC11c2Vyc10gc3BtIGNoYW5nZXMgaG91cmx5IGR1 ZSB0byB1bmV4cGVjdGVkIGVycm9yIG9mIGdldEFsbFRhc2tzU3RhdHVzZXMNCiANCiANCi0tLS0t IE9yaWdpbmFsIE1lc3NhZ2UgLS0tLS0NCj4gRnJvbTogImxpa2UgbWEiIDxsaWtlLm1hQGNzMmMu Y29tLmNuPg0KPiBUbzogZGV2ZWxAb3ZpcnQub3JnLCB1c2Vyc0BvdmlydC5vcmcNCj4gU2VudDog VHVlc2RheSwgSnVuZSAyMywgMjAxNSAxMToyNjoxNyBBTQ0KPiBTdWJqZWN0OiBbb3ZpcnQtdXNl cnNdIHNwbSBjaGFuZ2VzIGhvdXJseSBkdWUgdG8gdW5leHBlY3RlZCBlcnJvciBvZiBnZXRBbGxU YXNrc1N0YXR1c2VzDQo+IA0KPiBIZWxsbyBhbGwsDQo+IA0KPiBJbiBvdXIgb3ZpcnQgZW52aXJv bm1lbnQob1ZpcnQgMy40LjQpLCB3ZSBoYXZlIDIgaG9zdHMsIGFuZCB0aGUgc3BtIGNoYW5nZXMN Cj4gaG91cmx5IGJldHdlZW4gdGhlIDIgaG9zdHMuIEluIHRoZSBldmVudHMgb2YgRGF0YUNlbnRl cnMgdGhlcmUgaXMgZm9sbG93aW5nDQo+IG1lc3NhZ2UgZWFjaCBob3VyOg0KPiAnRGF0YSBDZW50 ZXIgaXMgYmVpbmcgaW5pdGlhbGl6ZWQsIHBsZWFzZSB3YWl0IGZvciBpbml0aWFsaXphdGlvbiB0 bw0KPiBjb21wbGV0ZScNCj4gDQo+IEluIHRoZSB2ZHNtLmxvZyBpIGZvdW5kIGZvbGxvd2luZyBl cnJvciBtZXNzYWdlOg0KPiBUaHJlYWQtMTM6OkVSUk9SOjoyMDE1LTA2LTIzDQo+IDEzOjM4OjM2 LDY2Nzo6dGFzazo6ODY2OjpUYXNrTWFuYWdlci5UYXNrOjooX3NldEVycm9yKQ0KPiBUYXNrPWBl NDUwZDdkNi03NzFkLTRjNTEtOTBiNy1jNmIxMGRhMzc4OTdgOjpVbmV4cGVjdGVkIGVycm9yDQo+ IFRyYWNlYmFjayAobW9zdCByZWNlbnQgY2FsbCBsYXN0KToNCj4gRmlsZSAiL3Vzci9zaGFyZS92 ZHNtL3N0b3JhZ2UvdGFzay5weSIsIGxpbmUgODczLCBpbiBfcnVuDQo+IHJldHVybiBmbigqYXJn cywgKiprYXJncykNCj4gRmlsZSAiL3Vzci9zaGFyZS92ZHNtL2xvZ1V0aWxzLnB5IiwgbGluZSA0 NSwgaW4gd3JhcHBlcg0KPiByZXMgPSBmKCphcmdzLCAqKmt3YXJncykNCj4gRmlsZSAiL3Vzci9z aGFyZS92ZHNtL3N0b3JhZ2UvaHNtLnB5IiwgbGluZSAyMTI1LCBpbiBnZXRBbGxUYXNrc1N0YXR1 c2VzDQo+IGFsbFRhc2tzU3RhdHVzID0gc3AuZ2V0QWxsVGFza3NTdGF0dXNlcygpDQo+IEZpbGUg Ii91c3Ivc2hhcmUvdmRzbS9zdG9yYWdlL3NlY3VyYWJsZS5weSIsIGxpbmUgNzMsIGluIHdyYXBw ZXINCj4gcmFpc2UgU2VjdXJlRXJyb3IoIlNlY3VyZWQgb2JqZWN0IGlzIG5vdCBpbiBzYWZlIHN0 YXRlIikNCj4gU2VjdXJlRXJyb3I6IFNlY3VyZWQgb2JqZWN0IGlzIG5vdCBpbiBzYWZlIHN0YXRl DQo+IA0KPiBUaGUgYXR0YWNobWVudCBpcyB2ZHNtLmxvZyBhbmQgZW5naW5lLmxvZy4gVGhlIGVy cm9yIG9jY3VyZWQgYXQgMjAxNS0wNi0yMw0KPiAxMzozOCBhbmQgMjAxNS0wNi0yMyAxNTozOCBp biB2ZHNtLmxvZy4NCj4gKGJ5IHRoZSB3YXksIGluIHRoZSBvdGhlciBob3N0IHRoZSBlcnJvciBv Y2N1cmVkIGF0IDIwMTUtMDYtMjMgMTI6MzggYW5kDQo+IDIwMTUtMDYtMjMgMTQ6MzggKQ0KPiAN Cj4gQW5kIGluIHRoZSBlbmdpbmUubG9nIGkgZm91bmQgZm9sbG93aW5nIGVycm9yIG1lc3NhZ2U6 DQo+IDIwMTUtMDYtMjMgMTM6Mzg6MzcsOTMzIEVSUk9SDQo+IFtvcmcub3ZpcnQuZW5naW5lLmNv cmUudmRzYnJva2VyLmlyc2Jyb2tlci5JcnNCcm9rZXJDb21tYW5kXQ0KPiAoRGVmYXVsdFF1YXJ0 elNjaGVkdWxlcl9Xb3JrZXItMzQpIFs3ODFiZWFlYV0gSXJzQnJva2VyOjpGYWlsZWQ6OlVwZGF0 ZVZNVkRTDQo+IGR1ZSB0bzogWG1sUnBjRXhjZXB0aW9uOiA8dHlwZSAnZXhjZXB0aW9ucy5Vbmlj b2RlRGVjb2RlRXJyb3InPjondXRmOCcgY29kZWMNCj4gY2FuJ3QgZGVjb2RlIGJ5dGUgMHhlYiBp biBwb3NpdGlvbiA3NDk6IGludmFsaWQgY29udGludWF0aW9uIGJ5dGUNCj4gDQo+IDIwMTUtMDYt MjMgMTM6Mzg6MzcsOTcxIEVSUk9SIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuYmxsLk92ZkRhdGFV cGRhdGVyXQ0KPiAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItMzQpIFs3ODFiZWFlYV0g RXhjZXB0aW9uIHdoaWxlIHRyeWluZyB0bw0KPiB1cGRhdGUgb3IgcmVtb3ZlIFZNcy9UZW1wbGF0 ZXMgb3ZmIGluIERhdGEgQ2VudGVyIERlZmF1bHQuOg0KPiBvcmcub3ZpcnQuZW5naW5lLmNvcmUu Y29tbW9uLmVycm9ycy5WZGNCTExFeGNlcHRpb246IFZkY0JMTEV4Y2VwdGlvbjoNCj4gamF2YS5s YW5nLnJlZmxlY3QuVW5kZWNsYXJlZFRocm93YWJsZUV4Y2VwdGlvbiAoRmFpbGVkIHdpdGggZXJy b3INCj4gVkRTX05FVFdPUktfRVJST1IgYW5kIGNvZGUgNTAyMikNCj4gYXQgb3JnLm92aXJ0LmVu Z2luZS5jb3JlLmJsbC5WZHNIYW5kbGVyLmhhbmRsZVZkc1Jlc3VsdChWZHNIYW5kbGVyLmphdmE6 MTE2KQ0KPiBbYmxsLmphcjpdDQo+IGF0DQo+IG9yZy5vdmlydC5lbmdpbmUuY29yZS5ibGwuVkRT QnJva2VyRnJvbnRlbmRJbXBsLlJ1blZkc0NvbW1hbmQoVkRTQnJva2VyRnJvbnRlbmRJbXBsLmph dmE6MzMpDQo+IFtibGwuamFyOl0NCj4gYXQNCj4gb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmJsbC5P dmZEYXRhVXBkYXRlci5leGVjdXRlVXBkYXRlVm1JblNwbUNvbW1hbmQoT3ZmRGF0YVVwZGF0ZXIu amF2YTozODMpDQo+IFtibGwuamFyOl0NCj4gYXQNCj4gb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmJs bC5PdmZEYXRhVXBkYXRlci5wZXJmb3JtT3ZmVXBkYXRlKE92ZkRhdGFVcGRhdGVyLmphdmE6MTYz KQ0KPiBbYmxsLmphcjpdDQo+IGF0DQo+IG9yZy5vdmlydC5lbmdpbmUuY29yZS5ibGwuT3ZmRGF0 YVVwZGF0ZXIudXBkYXRlT3ZmRm9yVm1zT2ZTdG9yYWdlUG9vbChPdmZEYXRhVXBkYXRlci5qYXZh OjEzNSkNCj4gW2JsbC5qYXI6XQ0KPiBhdA0KPiBvcmcub3ZpcnQuZW5naW5lLmNvcmUuYmxsLk92 ZkRhdGFVcGRhdGVyLm92ZlVwZGF0ZV90aW1lcihPdmZEYXRhVXBkYXRlci5qYXZhOjkzKQ0KPiBb YmxsLmphcjpdDQo+IGF0IHN1bi5yZWZsZWN0LkdlbmVyYXRlZE1ldGhvZEFjY2Vzc29yMjUwLmlu dm9rZShVbmtub3duIFNvdXJjZSkgWzoxLjcuMF82NV0NCj4gYXQNCj4gc3VuLnJlZmxlY3QuRGVs ZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbC5pbnZva2UoRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29y SW1wbC5qYXZhOjQzKQ0KPiBbcnQuamFyOjEuNy4wXzY1XQ0KPiBhdCBqYXZhLmxhbmcucmVmbGVj dC5NZXRob2QuaW52b2tlKE1ldGhvZC5qYXZhOjYwNikgW3J0LmphcjoxLjcuMF82NV0NCj4gYXQg b3JnLm92aXJ0LmVuZ2luZS5jb3JlLnV0aWxzLnRpbWVyLkpvYldyYXBwZXIuZXhlY3V0ZShKb2JX cmFwcGVyLmphdmE6NjApDQo+IFtzY2hlZHVsZXIuamFyOl0NCj4gYXQgb3JnLnF1YXJ0ei5jb3Jl LkpvYlJ1blNoZWxsLnJ1bihKb2JSdW5TaGVsbC5qYXZhOjIxMykgW3F1YXJ0ei5qYXI6XQ0KPiBh dA0KPiBvcmcucXVhcnR6LnNpbXBsLlNpbXBsZVRocmVhZFBvb2wkV29ya2VyVGhyZWFkLnJ1bihT aW1wbGVUaHJlYWRQb29sLmphdmE6NTU3KQ0KPiBbcXVhcnR6LmphcjpdDQo+IA0KPiBDYW4gYW55 b25lIGhlbHA/DQogDQpDYW4geW91IHBsZWFzZSBzaGFyZSB0aGUgcmVzdWx0IG9mIHRoZSBmb2xs b3dpbmcgb24gYm90aCBob3N0cyA/DQogDQo+IGVjaG8gJExBTkcNCiANCnRoYW5rcyANCiANCiAN Cj4gVGhhbmtzDQo+IA0KPiBsaWtlLm1hQGNzMmMuY29tLmNuDQo+IA0KPiBfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KPiBVc2VycyBtYWlsaW5nIGxpc3QN Cj4gVXNlcnNAb3ZpcnQub3JnDQo+IGh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0 aW5mby91c2Vycw0KPiANCg== ------=_001_NextPart870643344860_=---- Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <html><head><meta http-equiv=3D"content-type" content=3D"text/html; charse= t=3DUTF-8"><style>body { line-height: 1.5; }blockquote { margin-top: 0px; = margin-bottom: 0px; margin-left: 0.5em; }body { font-size: 10.5pt; font-fa= mily: =E5=BE=AE=E8=BD=AF=E9=9B=85=E9=BB=91; color: rgb(0, 0, 0); line-heig= ht: 1.5; }</style></head><body>=0A<div>Thank you for your reply.</div><div= div>But i think may be it's not the problem of locale,&nbsp;</div><div>bec= ause when i run 'vdsClient -s 0 getAllTasksStatuses' manually on the SPM h= ost, there is no problem(no errors in vdsm.log)</div>=0A<div><br></div><hr= style=3D"width: 210px; height: 1px;" color=3D"#b5c4df" size=3D"1" align= =3D"left">=0A<div><span><div style=3D"MARGIN: 10px; FONT-FAMILY: verdana; = FONT-SIZE: 10pt&quot;&gt;&lt;div&gt;like.ma(a)cs2c.com.cn&lt;/div&gt;&lt;/div&gt;&lt;/span&gt;&lt;/div&gt;=0A&lt;bloc= kquote style=3D"margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;">= <div>&nbsp;</div><div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;= padding:3.0pt 0cm 0cm 0cm"><div style=3D"PADDING-RIGHT: 8px; PADDING-LEFT:= 8px; FONT-SIZE: 12px;FONT-FAMILY:tahoma;COLOR:#000000; BACKGROUND: #efefe= f; PADDING-BOTTOM: 8px; PADDING-TOP: 8px"><div><b>From:</b>&nbsp;<a href= =3D"mailto:emesika@redhat.com">Eli Mesika</a></div><div><b>Date:</b>&nbsp;= 2015-06-23&nbsp;16:44</div><div><b>To:</b>&nbsp;<a href=3D"mailto:like.ma@= cs2c.com.cn">like ma</a></div><div><b>CC:</b>&nbsp;<a href=3D"mailto:devel= @ovirt.org">devel</a>; <a href=3D"mailto:users@ovirt.org">users</a></div><= div><b>Subject:</b>&nbsp;Re: [ovirt-users] spm changes hourly due to unexp= ected error of getAllTasksStatuses</div></div></div><div><div>&nbsp;</div>= =0A<div>&nbsp;</div>=0A<div>----- Original Message -----</div>=0A<div>&gt;= From: "like ma" &amp;lt;like.ma(a)cs2c.com.cn&amp;gt;&lt;/div&gt;=0A&lt;div&gt;&amp;gt; To: devel@o= virt.org, users(a)ovirt.org&lt;/div&gt;=0A&lt;div&gt;&amp;gt; Sent: Tuesday, June 23, 2015 1= 1:26:17 AM</div>=0A<div>&gt; Subject: [ovirt-users] spm changes hourly due= to unexpected error of getAllTasksStatuses</div>=0A<div>&gt; </div>=0A<di= v>&gt; Hello all,</div>=0A<div>&gt; </div>=0A<div>&gt; In our ovirt enviro= nment(oVirt 3.4.4), we have 2 hosts, and the spm changes</div>=0A<div>&gt;= hourly between the 2 hosts. In the events of DataCenters there is followi= ng</div>=0A<div>&gt; message each hour:</div>=0A<div>&gt; 'Data Center is = being initialized, please wait for initialization to</div>=0A<div>&gt; com= plete'</div>=0A<div>&gt; </div>=0A<div>&gt; In the vdsm.log i found follow= ing error message:</div>=0A<div>&gt; Thread-13::ERROR::2015-06-23</div>=0A= <div>&gt; 13:38:36,667::task::866::TaskManager.Task::(_setError)</div>=0A<= div>&gt; Task=3D`e450d7d6-771d-4c51-90b7-c6b10da37897`::Unexpected error</= div>=0A<div>&gt; Traceback (most recent call last):</div>=0A<div>&gt; File= "/usr/share/vdsm/storage/task.py", line 873, in _run</div>=0A<div>&gt; re= turn fn(*args, **kargs)</div>=0A<div>&gt; File "/usr/share/vdsm/logUtils.p= y", line 45, in wrapper</div>=0A<div>&gt; res =3D f(*args, **kwargs)</div>= =0A<div>&gt; File "/usr/share/vdsm/storage/hsm.py", line 2125, in getAllTa= sksStatuses</div>=0A<div>&gt; allTasksStatus =3D sp.getAllTasksStatuses()<= /div>=0A<div>&gt; File "/usr/share/vdsm/storage/securable.py", line 73, in= wrapper</div>=0A<div>&gt; raise SecureError("Secured object is not in saf= e state")</div>=0A<div>&gt; SecureError: Secured object is not in safe sta= te</div>=0A<div>&gt; </div>=0A<div>&gt; The attachment is vdsm.log and eng= ine.log. The error occured at 2015-06-23</div>=0A<div>&gt; 13:38 and 2015-= 06-23 15:38 in vdsm.log.</div>=0A<div>&gt; (by the way, in the other host = the error occured at 2015-06-23 12:38 and</div>=0A<div>&gt; 2015-06-23 14:= 38 )</div>=0A<div>&gt; </div>=0A<div>&gt; And in the engine.log i found fo= llowing error message:</div>=0A<div>&gt; 2015-06-23 13:38:37,933 ERROR</di= v>=0A<div>&gt; [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand= ]</div>=0A<div>&gt; (DefaultQuartzScheduler_Worker-34) [781beaea] IrsBroke= r::Failed::UpdateVMVDS</div>=0A<div>&gt; due to: XmlRpcException: &lt;type= 'exceptions.UnicodeDecodeError'&gt;:'utf8' codec</div>=0A<div>&gt; can't = decode byte 0xeb in position 749: invalid continuation byte</div>=0A<div>&= gt; </div>=0A<div>&gt; 2015-06-23 13:38:37,971 ERROR [org.ovirt.engine.cor= e.bll.OvfDataUpdater]</div>=0A<div>&gt; (DefaultQuartzScheduler_Worker-34)= [781beaea] Exception while trying to</div>=0A<div>&gt; update or remove V= Ms/Templates ovf in Data Center Default.:</div>=0A<div>&gt; org.ovirt.engi= ne.core.common.errors.VdcBLLException: VdcBLLException:</div>=0A<div>&gt; = java.lang.reflect.UndeclaredThrowableException (Failed with error</div>=0A= <div>&gt; VDS_NETWORK_ERROR and code 5022)</div>=0A<div>&gt; at org.ovirt.= engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:116)</div>=0A<d= iv>&gt; [bll.jar:]</div>=0A<div>&gt; at</div>=0A<div>&gt; org.ovirt.engine= .core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:3= 3)</div>=0A<div>&gt; [bll.jar:]</div>=0A<div>&gt; at</div>=0A<div>&gt; org= .ovirt.engine.core.bll.OvfDataUpdater.executeUpdateVmInSpmCommand(OvfDataU= pdater.java:383)</div>=0A<div>&gt; [bll.jar:]</div>=0A<div>&gt; at</div>= =0A<div>&gt; org.ovirt.engine.core.bll.OvfDataUpdater.performOvfUpdate(Ovf= DataUpdater.java:163)</div>=0A<div>&gt; [bll.jar:]</div>=0A<div>&gt; at</d= iv>=0A<div>&gt; org.ovirt.engine.core.bll.OvfDataUpdater.updateOvfForVmsOf= StoragePool(OvfDataUpdater.java:135)</div>=0A<div>&gt; [bll.jar:]</div>=0A= <div>&gt; at</div>=0A<div>&gt; org.ovirt.engine.core.bll.OvfDataUpdater.ov= fUpdate_timer(OvfDataUpdater.java:93)</div>=0A<div>&gt; [bll.jar:]</div>= =0A<div>&gt; at sun.reflect.GeneratedMethodAccessor250.invoke(Unknown Sour= ce) [:1.7.0_65]</div>=0A<div>&gt; at</div>=0A<div>&gt; sun.reflect.Delegat= ingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</div>= =0A<div>&gt; [rt.jar:1.7.0_65]</div>=0A<div>&gt; at java.lang.reflect.Meth= od.invoke(Method.java:606) [rt.jar:1.7.0_65]</div>=0A<div>&gt; at org.ovir= t.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60)</div>=0A<= div>&gt; [scheduler.jar:]</div>=0A<div>&gt; at org.quartz.core.JobRunShell= .run(JobRunShell.java:213) [quartz.jar:]</div>=0A<div>&gt; at</div>=0A<div= java:557)</div>=0A<div>&gt; [quartz.jar:]</div>=0A<div>&gt; </div>=0A<div>= &gt; Can anyone help?</div>=0A<div>&nbsp;</div>=0A<div>Can you please shar= e the result of the following on both hosts ?</div>=0A<div>&nbsp;</div>=0A= <div>&gt; echo $LANG</div>=0A<div>&nbsp;</div>=0A<div>thanks </div>=0A<div= iv>=0A<div>&gt; like.ma(a)cs2c.com.cn&lt;/div&gt;=0A&lt;div&gt;&amp;gt; </div>=0A<div>&gt; _= ______________________________________________</div>=0A<div>&gt; Users mai= ling list</div>=0A<div>&gt; Users(a)ovirt.org&lt;/div&gt;=0A&lt;div&gt;&amp;gt; http://lists= .ovirt.org/mailman/listinfo/users</div>=0A<div>&gt; </div>=0A</div></block= quote>=0A</body></html> ------=_001_NextPart870643344860_=------