--_000_ABC69195C9484895AAFF1900F54BBE43unilu_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SGksDQoNCkkgY2hlY2tlZCB0aGUgbG9ncyBvbiBteSBoeXBlcnZpc29yIHRoYXQgY29udGFpbnMg YWxzbyB0aGUgb3ZlcnQtZW5naW5lIChzZWxmLWhvc3RlZCkgYW5kIEkgc2VlIHN0cmFuZ2UgdW5r bm93biBsaWJ2aXJ0ZXJyb3JzIHRoYXQgY29tZSBwZXJpb2RpY2FsbHkgaW4gdGhlIHZkc20ubG9n IGZpbGUuIFRoZSBzdG9yYWdlIGlzIGdsdXN0ZXJGUyBydW5uaW5nIG9uIHRoZSBoeXBlcnZpc29y IGFzIHdlbGwsIG9uZSBORlMgZXhwb3J0IGRvbWFpbiBhbmQgYW4gSVNPIGRvbWFpbi4gQSBORlMg ZG9tYWluIGZyb20gYW5vdGhlciBwbGFjZSBpcyBpbiBtYWludGVuYW5jZSBtb2RlLg0KDQpJIGFt IHJ1bm5pbmcgb1ZpcnQgMy41LjMuDQoNClRoYW5rIHlvdSBmb3IgYW55IHBvaW50ZXJzIGFzIHRv IHdoZXJlIHRvIHN0YXJ0IGZpeGluZyB0aGlzIGlzc3VlLg0KDQrigJQgbG9nIGV4Y2VycHQgLS0N Cg0KVGhyZWFkLTE5NDc2NDE6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMxLDM5ODo6c3RvbXBS ZWFjdG9yOjoxNjM6OnlhanNvbnJwYy5TdG9tcFNlcnZlcjo6KHNlbmQpIFNlbmRpbmcgcmVzcG9u c2UNClRocmVhZC04MTA4OjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozMSw0MTA6OmxpYnZpcnRj b25uZWN0aW9uOjoxNDM6OnJvb3Q6Oih3cmFwcGVyKSBVbmtub3duIGxpYnZpcnRlcnJvcjogZWNv ZGU6IDgwIGVkb206IDIwIGxldmVsOiAyIG1lc3NhZ2U6IG1ldGFkYXRhIG5vdCBmb3VuZDogUmVx dWVzdGVkIG1ldGFkYXRhIGVsZW1lbnQgaXMgbm90IHByZXNlbnQNCkR1bW15LTE4OTUyNjA6OkRF QlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMxLDQ3Nzo6c3RvcmFnZV9tYWlsYm94Ojo3MzE6OlN0b3Jh Z2UuTWlzYy5leGNDbWQ6OihfY2hlY2tGb3JNYWlsKSBkZCBpZj0vcmhldi9kYXRhLWNlbnRlci8w MDAwMDAwMi0wMDAyLTAwMDItMDAwMi0wMDAwMDAwMDAzZDUvbWFzdGVyc2QvZG9tX21kL2luYm94 IGlmbGFnPWRpcmVjdCxmdWxsYmxvY2sgY291bnQ9MSBicz0xMDI0MDAwIChjd2QgTm9uZSkNCkR1 bW15LTE4OTUyNjA6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMxLDUwMTo6c3RvcmFnZV9tYWls Ym94Ojo3MzE6OlN0b3JhZ2UuTWlzYy5leGNDbWQ6OihfY2hlY2tGb3JNYWlsKSBTVUNDRVNTOiA8 ZXJyPiA9ICcxKzAgcmVjb3JkcyBpblxuMSswIHJlY29yZHMgb3V0XG4xMDI0MDAwIGJ5dGVzICgx LjAgTUIpIGNvcGllZCwgMC4wMDMzMTI3OCBzLCAzMDkgTUIvc1xuJzsgPHJjPiA9IDANClRocmVh ZC03OTEzOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozMiwyOTg6OmxpYnZpcnRjb25uZWN0aW9u OjoxNDM6OnJvb3Q6Oih3cmFwcGVyKSBVbmtub3duIGxpYnZpcnRlcnJvcjogZWNvZGU6IDgwIGVk b206IDIwIGxldmVsOiAyIG1lc3NhZ2U6IG1ldGFkYXRhIG5vdCBmb3VuZDogUmVxdWVzdGVkIG1l dGFkYXRhIGVsZW1lbnQgaXMgbm90IHByZXNlbnQNClRocmVhZC01NjgyOjpERUJVRzo6MjAxNS0x MS0wMyAwODo0NzozMiw0MTc6OmxpYnZpcnRjb25uZWN0aW9uOjoxNDM6OnJvb3Q6Oih3cmFwcGVy KSBVbmtub3duIGxpYnZpcnRlcnJvcjogZWNvZGU6IDgwIGVkb206IDIwIGxldmVsOiAyIG1lc3Nh Z2U6IG1ldGFkYXRhIG5vdCBmb3VuZDogUmVxdWVzdGVkIG1ldGFkYXRhIGVsZW1lbnQgaXMgbm90 IHByZXNlbnQNCkRldGVjdG9yIHRocmVhZDo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzIsNTkx Ojpwcm90b2NvbGRldGVjdG9yOjoxODc6OnZkcy5NdWx0aVByb3RvY29sQWNjZXB0b3I6OihfYWRk X2Nvbm5lY3Rpb24pIEFkZGluZyBjb25uZWN0aW9uIGZyb20gMTI3LjAuMC4xOjQ0NjcxDQpEZXRl Y3RvciB0aHJlYWQ6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMyLDU5ODo6cHJvdG9jb2xkZXRl Y3Rvcjo6MjAxOjp2ZHMuTXVsdGlQcm90b2NvbEFjY2VwdG9yOjooX3JlbW92ZV9jb25uZWN0aW9u KSBDb25uZWN0aW9uIHJlbW92ZWQgZnJvbSAxMjcuMC4wLjE6NDQ2NzENCkRldGVjdG9yIHRocmVh ZDo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzIsNTk5Ojpwcm90b2NvbGRldGVjdG9yOjoyNDc6 OnZkcy5NdWx0aVByb3RvY29sQWNjZXB0b3I6OihfaGFuZGxlX2Nvbm5lY3Rpb25fcmVhZCkgRGV0 ZWN0ZWQgcHJvdG9jb2wgeG1sIGZyb20gMTI3LjAuMC4xOjQ0NjcxDQpEZXRlY3RvciB0aHJlYWQ6 OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMyLDU5OTo6QmluZGluZ1hNTFJQQzo6MTE3Mzo6WG1s RGV0ZWN0b3I6OihoYW5kbGVTb2NrZXQpIHhtbCBvdmVyIGh0dHAgZGV0ZWN0ZWQgZnJvbSAoJzEy Ny4wLjAuMScsIDQ0NjcxKQ0KVGhyZWFkLTE5NDc2NDI6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3 OjMyLDYwMjo6dGFzazo6NTk1OjpTdG9yYWdlLlRhc2tNYW5hZ2VyLlRhc2s6OihfdXBkYXRlU3Rh dGUpIFRhc2s9YDFkOTlhMTY2LWNiOWEtNDAyNS04MjExLWE0OGUyMTBiNTIzNGA6Om1vdmluZyBm cm9tIHN0YXRlIGluaXQgLT4gc3RhdGUgcHJlcGFyaW5nDQpUaHJlYWQtMTk0NzY0Mjo6SU5GTzo6 MjAxNS0xMS0wMyAwODo0NzozMiw2MDM6OmxvZ1V0aWxzOjo0NDo6ZGlzcGF0Y2hlcjo6KHdyYXBw ZXIpIFJ1biBhbmQgcHJvdGVjdDogcmVwb1N0YXRzKG9wdGlvbnM9Tm9uZSkNClRocmVhZC0xOTQ3 NjQyOjpJTkZPOjoyMDE1LTExLTAzIDA4OjQ3OjMyLDYwMzo6bG9nVXRpbHM6OjQ3OjpkaXNwYXRj aGVyOjood3JhcHBlcikgUnVuIGFuZCBwcm90ZWN0OiByZXBvU3RhdHMsIFJldHVybiByZXNwb25z ZToge3UnZGU5ZWI3MzctNjkxZi00NjIyLTkwNzAtODkxNTMxZDU5OWEwJzogeydjb2RlJzogMCwg J2FjdHVhbCc6IFRydWUsICd2ZXJzaW9uJzogMCwgJ2FjcXVpcmVkJzogVHJ1ZSwgJ2RlbGF5Jzog JzAuMDAwMzczNjEzJywgJ2xhc3RDaGVjayc6ICcyLjUnLCAndmFsaWQnOiBUcnVlfSwgdSdmZTRm ZDE5YS04NzE0LTQ0ZTAtYWU0MS02NjNhNGI2MmRhN2EnOiB7J2NvZGUnOiAwLCAnYWN0dWFsJzog VHJ1ZSwgJ3ZlcnNpb24nOiAwLCAnYWNxdWlyZWQnOiBUcnVlLCAnZGVsYXknOiAnMC4wMDA0MDk0 NDYnLCAnbGFzdENoZWNrJzogJzYuNCcsICd2YWxpZCc6IFRydWV9LCB1JzgyNTNhODliLTY1MWUt NGZmNC04NjViLTU3YWRlZjA1ZDM4Myc6IHsnY29kZSc6IDAsICdhY3R1YWwnOiBUcnVlLCAndmVy c2lvbic6IDMsICdhY3F1aXJlZCc6IFRydWUsICdkZWxheSc6ICcwLjAwMDUyMDY3MScsICdsYXN0 Q2hlY2snOiAnMS44JywgJ3ZhbGlkJzogVHJ1ZX0sICdiMThlYjI5ZS04YmIxLTQ1YjktYTYwZS1h OGUwNzIxMGUwNjYnOiB7J2NvZGUnOiAwLCAnYWN0dWFsJzogVHJ1ZSwgJ3ZlcnNpb24nOiAzLCAn YWNxdWlyZWQnOiBUcnVlLCAnZGVsYXknOiAnMC4wMDA0MjQ0NDUnLCAnbGFzdENoZWNrJzogJzYu NScsICd2YWxpZCc6IFRydWV9fQ0KVGhyZWFkLTE5NDc2NDI6OkRFQlVHOjoyMDE1LTExLTAzIDA4 OjQ3OjMyLDYwMzo6dGFzazo6MTE5MTo6U3RvcmFnZS5UYXNrTWFuYWdlci5UYXNrOjoocHJlcGFy ZSkgVGFzaz1gMWQ5OWExNjYtY2I5YS00MDI1LTgyMTEtYTQ4ZTIxMGI1MjM0YDo6ZmluaXNoZWQ6 IHt1J2RlOWViNzM3LTY5MWYtNDYyMi05MDcwLTg5MTUzMWQ1OTlhMCc6IHsnY29kZSc6IDAsICdh Y3R1YWwnOiBUcnVlLCAndmVyc2lvbic6IDAsICdhY3F1aXJlZCc6IFRydWUsICdkZWxheSc6ICcw LjAwMDM3MzYxMycsICdsYXN0Q2hlY2snOiAnMi41JywgJ3ZhbGlkJzogVHJ1ZX0sIHUnZmU0ZmQx OWEtODcxNC00NGUwLWFlNDEtNjYzYTRiNjJkYTdhJzogeydjb2RlJzogMCwgJ2FjdHVhbCc6IFRy dWUsICd2ZXJzaW9uJzogMCwgJ2FjcXVpcmVkJzogVHJ1ZSwgJ2RlbGF5JzogJzAuMDAwNDA5NDQ2 JywgJ2xhc3RDaGVjayc6ICc2LjQnLCAndmFsaWQnOiBUcnVlfSwgdSc4MjUzYTg5Yi02NTFlLTRm ZjQtODY1Yi01N2FkZWYwNWQzODMnOiB7J2NvZGUnOiAwLCAnYWN0dWFsJzogVHJ1ZSwgJ3ZlcnNp b24nOiAzLCAnYWNxdWlyZWQnOiBUcnVlLCAnZGVsYXknOiAnMC4wMDA1MjA2NzEnLCAnbGFzdENo ZWNrJzogJzEuOCcsICd2YWxpZCc6IFRydWV9LCAnYjE4ZWIyOWUtOGJiMS00NWI5LWE2MGUtYThl MDcyMTBlMDY2Jzogeydjb2RlJzogMCwgJ2FjdHVhbCc6IFRydWUsICd2ZXJzaW9uJzogMywgJ2Fj cXVpcmVkJzogVHJ1ZSwgJ2RlbGF5JzogJzAuMDAwNDI0NDQ1JywgJ2xhc3RDaGVjayc6ICc2LjUn LCAndmFsaWQnOiBUcnVlfX0NClRocmVhZC0xOTQ3NjQyOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0 NzozMiw2MDM6OnRhc2s6OjU5NTo6U3RvcmFnZS5UYXNrTWFuYWdlci5UYXNrOjooX3VwZGF0ZVN0 YXRlKSBUYXNrPWAxZDk5YTE2Ni1jYjlhLTQwMjUtODIxMS1hNDhlMjEwYjUyMzRgOjptb3Zpbmcg ZnJvbSBzdGF0ZSBwcmVwYXJpbmcgLT4gc3RhdGUgZmluaXNoZWQNClRocmVhZC0xOTQ3NjQyOjpE RUJVRzo6MjAxNS0xMS0wMyAwODo0NzozMiw2MDQ6OnJlc291cmNlTWFuYWdlcjo6OTQwOjpTdG9y YWdlLlJlc291cmNlTWFuYWdlci5Pd25lcjo6KHJlbGVhc2VBbGwpIE93bmVyLnJlbGVhc2VBbGwg cmVxdWVzdHMge30gcmVzb3VyY2VzIHt9DQpUaHJlYWQtMTk0NzY0Mjo6REVCVUc6OjIwMTUtMTEt MDMgMDg6NDc6MzIsNjA0OjpyZXNvdXJjZU1hbmFnZXI6Ojk3Nzo6U3RvcmFnZS5SZXNvdXJjZU1h bmFnZXIuT3duZXI6OihjYW5jZWxBbGwpIE93bmVyLmNhbmNlbEFsbCByZXF1ZXN0cyB7fQ0KVGhy ZWFkLTE5NDc2NDI6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMyLDYwNDo6dGFzazo6OTkzOjpT dG9yYWdlLlRhc2tNYW5hZ2VyLlRhc2s6OihfZGVjcmVmKSBUYXNrPWAxZDk5YTE2Ni1jYjlhLTQw MjUtODIxMS1hNDhlMjEwYjUyMzRgOjpyZWYgMCBhYm9ydGluZyBGYWxzZQ0KVGhyZWFkLTYzNDg6 OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMzLDI2MTo6bGlidmlydGNvbm5lY3Rpb246OjE0Mzo6 cm9vdDo6KHdyYXBwZXIpIFVua25vd24gbGlidmlydGVycm9yOiBlY29kZTogODAgZWRvbTogMjAg bGV2ZWw6IDIgbWVzc2FnZTogbWV0YWRhdGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0YWRhdGEg ZWxlbWVudCBpcyBub3QgcHJlc2VudA0KVGhyZWFkLTcyODY6OkRFQlVHOjoyMDE1LTExLTAzIDA4 OjQ3OjMzLDQ2Mjo6bGlidmlydGNvbm5lY3Rpb246OjE0Mzo6cm9vdDo6KHdyYXBwZXIpIFVua25v d24gbGlidmlydGVycm9yOiBlY29kZTogODAgZWRvbTogMjAgbGV2ZWw6IDIgbWVzc2FnZTogbWV0 YWRhdGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0YWRhdGEgZWxlbWVudCBpcyBub3QgcHJlc2Vu dA0KRHVtbXktMTg5NTI2MDo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzMsNTE0OjpzdG9yYWdl X21haWxib3g6OjczMTo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KF9jaGVja0Zvck1haWwpIGRkIGlm PS9yaGV2L2RhdGEtY2VudGVyLzAwMDAwMDAyLTAwMDItMDAwMi0wMDAyLTAwMDAwMDAwMDNkNS9t YXN0ZXJzZC9kb21fbWQvaW5ib3ggaWZsYWc9ZGlyZWN0LGZ1bGxibG9jayBjb3VudD0xIGJzPTEw MjQwMDAgKGN3ZCBOb25lKQ0KRHVtbXktMTg5NTI2MDo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6 MzMsNTQwOjpzdG9yYWdlX21haWxib3g6OjczMTo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KF9jaGVj a0Zvck1haWwpIFNVQ0NFU1M6IDxlcnI+ID0gJzErMCByZWNvcmRzIGluXG4xKzAgcmVjb3JkcyBv dXRcbjEwMjQwMDAgYnl0ZXMgKDEuMCBNQikgY29waWVkLCAwLjAwMzg1OTAxIHMsIDI2NSBNQi9z XG4nOyA8cmM+ID0gMA0KVGhyZWFkLTc2Mjc6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMzLDkz ODo6bGlidmlydGNvbm5lY3Rpb246OjE0Mzo6cm9vdDo6KHdyYXBwZXIpIFVua25vd24gbGlidmly dGVycm9yOiBlY29kZTogODAgZWRvbTogMjAgbGV2ZWw6IDIgbWVzc2FnZTogbWV0YWRhdGEgbm90 IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0YWRhdGEgZWxlbWVudCBpcyBub3QgcHJlc2VudA0KVGhyZWFk LTc5NTE6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMzLDkzODo6bGlidmlydGNvbm5lY3Rpb246 OjE0Mzo6cm9vdDo6KHdyYXBwZXIpIFVua25vd24gbGlidmlydGVycm9yOiBlY29kZTogODAgZWRv bTogMjAgbGV2ZWw6IDIgbWVzc2FnZTogbWV0YWRhdGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0 YWRhdGEgZWxlbWVudCBpcyBub3QgcHJlc2VudA0KVGhyZWFkLTM4ODI6OkRFQlVHOjoyMDE1LTEx LTAzIDA4OjQ3OjMzLDk0MDo6bGlidmlydGNvbm5lY3Rpb246OjE0Mzo6cm9vdDo6KHdyYXBwZXIp IFVua25vd24gbGlidmlydGVycm9yOiBlY29kZTogODAgZWRvbTogMjAgbGV2ZWw6IDIgbWVzc2Fn ZTogbWV0YWRhdGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0YWRhdGEgZWxlbWVudCBpcyBub3Qg cHJlc2VudA0KVGhyZWFkLTc5Njc6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMzLDk0MDo6bGli dmlydGNvbm5lY3Rpb246OjE0Mzo6cm9vdDo6KHdyYXBwZXIpIFVua25vd24gbGlidmlydGVycm9y OiBlY29kZTogODAgZWRvbTogMjAgbGV2ZWw6IDIgbWVzc2FnZTogbWV0YWRhdGEgbm90IGZvdW5k OiBSZXF1ZXN0ZWQgbWV0YWRhdGEgZWxlbWVudCBpcyBub3QgcHJlc2VudA0KVGhyZWFkLTc4OTk6 OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMzLDk1MTo6bGlidmlydGNvbm5lY3Rpb246OjE0Mzo6 cm9vdDo6KHdyYXBwZXIpIFVua25vd24gbGlidmlydGVycm9yOiBlY29kZTogODAgZWRvbTogMjAg bGV2ZWw6IDIgbWVzc2FnZTogbWV0YWRhdGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0YWRhdGEg ZWxlbWVudCBpcyBub3QgcHJlc2VudA0KVk0gQ2hhbm5lbHMgTGlzdGVuZXI6OkRFQlVHOjoyMDE1 LTExLTAzIDA4OjQ3OjM0LDM4Mzo6dm1jaGFubmVsczo6OTY6OnZkczo6KF9oYW5kbGVfdGltZW91 dHMpIFRpbWVvdXQgb24gZmlsZW5vIDEzMy4NClZNIENoYW5uZWxzIExpc3RlbmVyOjpERUJVRzo6 MjAxNS0xMS0wMyAwODo0NzozNCwzODM6OnZtY2hhbm5lbHM6Ojk2Ojp2ZHM6OihfaGFuZGxlX3Rp bWVvdXRzKSBUaW1lb3V0IG9uIGZpbGVubyAxMzUuDQpWTSBDaGFubmVscyBMaXN0ZW5lcjo6REVC VUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzgzOjp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRs ZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTM2Lg0KVk0gQ2hhbm5lbHMgTGlzdGVuZXI6 OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjM0LDM4Mzo6dm1jaGFubmVsczo6OTY6OnZkczo6KF9o YW5kbGVfdGltZW91dHMpIFRpbWVvdXQgb24gZmlsZW5vIDE0Ni4NClZNIENoYW5uZWxzIExpc3Rl bmVyOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozNCwzODQ6OnZtY2hhbm5lbHM6Ojk2Ojp2ZHM6 OihfaGFuZGxlX3RpbWVvdXRzKSBUaW1lb3V0IG9uIGZpbGVubyAxNjAuDQpWTSBDaGFubmVscyBM aXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg0Ojp2bWNoYW5uZWxzOjo5Njo6 dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTYxLg0KVk0gQ2hhbm5l bHMgTGlzdGVuZXI6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjM0LDM4NDo6dm1jaGFubmVsczo6 OTY6OnZkczo6KF9oYW5kbGVfdGltZW91dHMpIFRpbWVvdXQgb24gZmlsZW5vIDE2Ny4NClZNIENo YW5uZWxzIExpc3RlbmVyOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozNCwzODQ6OnZtY2hhbm5l bHM6Ojk2Ojp2ZHM6OihfaGFuZGxlX3RpbWVvdXRzKSBUaW1lb3V0IG9uIGZpbGVubyAxNzEuDQpW TSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg0Ojp2bWNo YW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTcy Lg0KVk0gQ2hhbm5lbHMgTGlzdGVuZXI6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjM0LDM4NDo6 dm1jaGFubmVsczo6OTY6OnZkczo6KF9oYW5kbGVfdGltZW91dHMpIFRpbWVvdXQgb24gZmlsZW5v IDE1OS4NClZNIENoYW5uZWxzIExpc3RlbmVyOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozNCwz ODU6OnZtY2hhbm5lbHM6Ojk2Ojp2ZHM6OihfaGFuZGxlX3RpbWVvdXRzKSBUaW1lb3V0IG9uIGZp bGVubyAxODkuDQpWTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6 MzQsMzg1Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBv biBmaWxlbm8gMTkwLg0KVk0gQ2hhbm5lbHMgTGlzdGVuZXI6OkRFQlVHOjoyMDE1LTExLTAzIDA4 OjQ3OjM0LDM4NTo6dm1jaGFubmVsczo6OTY6OnZkczo6KF9oYW5kbGVfdGltZW91dHMpIFRpbWVv dXQgb24gZmlsZW5vIDE5NS4NClZNIENoYW5uZWxzIExpc3RlbmVyOjpERUJVRzo6MjAxNS0xMS0w MyAwODo0NzozNCwzODU6OnZtY2hhbm5lbHM6Ojk2Ojp2ZHM6OihfaGFuZGxlX3RpbWVvdXRzKSBU aW1lb3V0IG9uIGZpbGVubyAxOTcuDQpWTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUt MTEtMDMgMDg6NDc6MzQsMzg1Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0 cykgVGltZW91dCBvbiBmaWxlbm8gMTk4Lg0KVk0gQ2hhbm5lbHMgTGlzdGVuZXI6OkRFQlVHOjoy MDE1LTExLTAzIDA4OjQ3OjM0LDM4NTo6dm1jaGFubmVsczo6OTY6OnZkczo6KF9oYW5kbGVfdGlt ZW91dHMpIFRpbWVvdXQgb24gZmlsZW5vIDIxMy4NClZNIENoYW5uZWxzIExpc3RlbmVyOjpERUJV Rzo6MjAxNS0xMS0wMyAwODo0NzozNCwzODY6OnZtY2hhbm5lbHM6Ojk2Ojp2ZHM6OihfaGFuZGxl X3RpbWVvdXRzKSBUaW1lb3V0IG9uIGZpbGVubyAyMTUuDQpWTSBDaGFubmVscyBMaXN0ZW5lcjo6 REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg2Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hh bmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gOTguDQpWTSBDaGFubmVscyBMaXN0ZW5l cjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg2Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjoo X2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTI1Lg0KSnNvblJwYyAoU3RvbXBS ZWFjdG9yKTo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsNDEyOjpzdG9tcFJlYWN0b3I6Ojk4 OjpCcm9rZXIuU3RvbXBBZGFwdGVyOjooaGFuZGxlX2ZyYW1lKSBIYW5kbGluZyBtZXNzYWdlIDxT dG9tcEZyYW1lIGNvbW1hbmQ9J1NFTkQnPg0KSnNvblJwY1NlcnZlcjo6REVCVUc6OjIwMTUtMTEt MDMgMDg6NDc6MzQsNDEzOjpfX2luaXRfXzo6NTMwOjpqc29ucnBjLkpzb25ScGNTZXJ2ZXI6Oihz ZXJ2ZV9yZXF1ZXN0cykgV2FpdGluZyBmb3IgcmVxdWVzdA0KVGhyZWFkLTE5NDc2NDM6OkRFQlVH OjoyMDE1LTExLTAzIDA4OjQ3OjM0LDQzNzo6c3RvbXBSZWFjdG9yOjoxNjM6OnlhanNvbnJwYy5T dG9tcFNlcnZlcjo6KHNlbmQpIFNlbmRpbmcgcmVzcG9uc2UNClRocmVhZC03NjEzOjpERUJVRzo6 MjAxNS0xMS0wMyAwODo0NzozNCw3MTM6OmxpYnZpcnRjb25uZWN0aW9uOjoxNDM6OnJvb3Q6Oih3 cmFwcGVyKSBVbmtub3duIGxpYnZpcnRlcnJvcjogZWNvZGU6IDgwIGVkb206IDIwIGxldmVsOiAy IG1lc3NhZ2U6IG1ldGFkYXRhIG5vdCBmb3VuZDogUmVxdWVzdGVkIG1ldGFkYXRhIGVsZW1lbnQg aXMgbm90IHByZXNlbnQNClRocmVhZC02MzkzOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozNCw3 MTM6OmxpYnZpcnRjb25uZWN0aW9uOjoxNDM6OnJvb3Q6Oih3cmFwcGVyKSBVbmtub3duIGxpYnZp cnRlcnJvcjogZWNvZGU6IDgwIGVkb206IDIwIGxldmVsOiAyIG1lc3NhZ2U6IG1ldGFkYXRhIG5v dCBmb3VuZDogUmVxdWVzdGVkIG1ldGFkYXRhIGVsZW1lbnQgaXMgbm90IHByZXNlbnQNClRocmVh ZC0yMjY5NDE6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjM1LDUxMTo6dGFzazo6NTk1OjpTdG9y YWdlLlRhc2tNYW5hZ2VyLlRhc2s6OihfdXBkYXRlU3RhdGUpIFRhc2s9YGJkZjI2NDAxLTMyNGMt NDIyMC05MDM0LTE5YzdkODE2ZjY0MmA6Om1vdmluZyBmcm9tIHN0YXRlIGluaXQgLT4gc3RhdGUg cHJlcGFyaW5nDQpUaHJlYWQtNDcwOTo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzUsNTExOjp0 YXNrOjo1OTU6OlN0b3JhZ2UuVGFza01hbmFnZXIuVGFzazo6KF91cGRhdGVTdGF0ZSkgVGFzaz1g MGFjMmY4NTQtNDFlMy00NDI3LWEwZjQtNWVhYTE4NDJlMjEyYDo6bW92aW5nIGZyb20gc3RhdGUg aW5pdCAtPiBzdGF0ZSBwcmVwYXJpbmcNCg0KLS0NCg0KQ2hyaXN0b3BoZQ0KDQoNCg== --_000_ABC69195C9484895AAFF1900F54BBE43unilu_ Content-Type: text/html; charset="utf-8" Content-ID: <05E547F58ADC464D9D0F9ACEF203BCD2@uni.lux> Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5IHN0eWxlPSJ3b3JkLXdy YXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgLXdlYmtpdC1saW5lLWJy ZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KSGksDQo8ZGl2IGNsYXNzPSIiPjxi ciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5JIGNoZWNrZWQgdGhlIGxvZ3Mgb24g bXkgaHlwZXJ2aXNvciB0aGF0IGNvbnRhaW5zIGFsc28gdGhlIG92ZXJ0LWVuZ2luZSAoc2VsZi1o b3N0ZWQpIGFuZCBJIHNlZSBzdHJhbmdlIHVua25vd24gbGlidmlydGVycm9ycyB0aGF0IGNvbWUg cGVyaW9kaWNhbGx5IGluIHRoZSB2ZHNtLmxvZyBmaWxlLiBUaGUgc3RvcmFnZSBpcyBnbHVzdGVy RlMgcnVubmluZyBvbiB0aGUgaHlwZXJ2aXNvciBhcyB3ZWxsLCBvbmUgTkZTIGV4cG9ydA0KIGRv bWFpbiBhbmQgYW4gSVNPIGRvbWFpbi4gQSBORlMgZG9tYWluIGZyb20gYW5vdGhlciBwbGFjZSBp cyBpbiBtYWludGVuYW5jZSBtb2RlLjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+ DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+SSBhbSBydW5uaW5nIG9WaXJ0IDMuNS4zLjwvZGl2Pg0K PGRpdiBjbGFzcz0iIj48YnIgY2xhc3M9IiI+DQo8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhhbmsg eW91IGZvciBhbnkgcG9pbnRlcnMgYXMgdG8gd2hlcmUgdG8gc3RhcnQgZml4aW5nIHRoaXMgaXNz dWUuPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGRpdiBjbGFz cz0iIj7igJQgbG9nIGV4Y2VycHQgLS08L2Rpdj4NCjxkaXYgY2xhc3M9IiI+DQo8ZGl2IGNsYXNz PSIiPjxiciBjbGFzcz0iIj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj4NCjxkaXYgY2xhc3M9IiI+ VGhyZWFkLTE5NDc2NDE6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMxLDM5ODo6c3RvbXBSZWFj dG9yOjoxNjM6OnlhanNvbnJwYy5TdG9tcFNlcnZlcjo6KHNlbmQpIFNlbmRpbmcgcmVzcG9uc2U8 L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhyZWFkLTgxMDg6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3 OjMxLDQxMDo6bGlidmlydGNvbm5lY3Rpb246OjE0Mzo6cm9vdDo6KHdyYXBwZXIpIFVua25vd24g bGlidmlydGVycm9yOiBlY29kZTogODAgZWRvbTogMjAgbGV2ZWw6IDIgbWVzc2FnZTogbWV0YWRh dGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0YWRhdGEgZWxlbWVudCBpcyBub3QgcHJlc2VudDwv ZGl2Pg0KPGRpdiBjbGFzcz0iIj5EdW1teS0xODk1MjYwOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0 NzozMSw0Nzc6OnN0b3JhZ2VfbWFpbGJveDo6NzMxOjpTdG9yYWdlLk1pc2MuZXhjQ21kOjooX2No ZWNrRm9yTWFpbCkgZGQgaWY9L3JoZXYvZGF0YS1jZW50ZXIvMDAwMDAwMDItMDAwMi0wMDAyLTAw MDItMDAwMDAwMDAwM2Q1L21hc3RlcnNkL2RvbV9tZC9pbmJveCBpZmxhZz1kaXJlY3QsZnVsbGJs b2NrIGNvdW50PTEgYnM9MTAyNDAwMCAoY3dkIE5vbmUpPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPkR1 bW15LTE4OTUyNjA6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMxLDUwMTo6c3RvcmFnZV9tYWls Ym94Ojo3MzE6OlN0b3JhZ2UuTWlzYy5leGNDbWQ6OihfY2hlY2tGb3JNYWlsKSBTVUNDRVNTOiAm bHQ7ZXJyJmd0OyA9ICcxJiM0MzswIHJlY29yZHMgaW5cbjEmIzQzOzAgcmVjb3JkcyBvdXRcbjEw MjQwMDAgYnl0ZXMgKDEuMCBNQikgY29waWVkLCAwLjAwMzMxMjc4IHMsIDMwOSBNQi9zXG4nOyAm bHQ7cmMmZ3Q7ID0gMDwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaHJlYWQtNzkxMzo6REVCVUc6OjIw MTUtMTEtMDMgMDg6NDc6MzIsMjk4OjpsaWJ2aXJ0Y29ubmVjdGlvbjo6MTQzOjpyb290Ojood3Jh cHBlcikgVW5rbm93biBsaWJ2aXJ0ZXJyb3I6IGVjb2RlOiA4MCBlZG9tOiAyMCBsZXZlbDogMiBt ZXNzYWdlOiBtZXRhZGF0YSBub3QgZm91bmQ6IFJlcXVlc3RlZCBtZXRhZGF0YSBlbGVtZW50IGlz IG5vdCBwcmVzZW50PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRocmVhZC01NjgyOjpERUJVRzo6MjAx NS0xMS0wMyAwODo0NzozMiw0MTc6OmxpYnZpcnRjb25uZWN0aW9uOjoxNDM6OnJvb3Q6Oih3cmFw cGVyKSBVbmtub3duIGxpYnZpcnRlcnJvcjogZWNvZGU6IDgwIGVkb206IDIwIGxldmVsOiAyIG1l c3NhZ2U6IG1ldGFkYXRhIG5vdCBmb3VuZDogUmVxdWVzdGVkIG1ldGFkYXRhIGVsZW1lbnQgaXMg bm90IHByZXNlbnQ8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+RGV0ZWN0b3IgdGhyZWFkOjpERUJVRzo6 MjAxNS0xMS0wMyAwODo0NzozMiw1OTE6OnByb3RvY29sZGV0ZWN0b3I6OjE4Nzo6dmRzLk11bHRp UHJvdG9jb2xBY2NlcHRvcjo6KF9hZGRfY29ubmVjdGlvbikgQWRkaW5nIGNvbm5lY3Rpb24gZnJv bSAxMjcuMC4wLjE6NDQ2NzE8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+RGV0ZWN0b3IgdGhyZWFkOjpE RUJVRzo6MjAxNS0xMS0wMyAwODo0NzozMiw1OTg6OnByb3RvY29sZGV0ZWN0b3I6OjIwMTo6dmRz Lk11bHRpUHJvdG9jb2xBY2NlcHRvcjo6KF9yZW1vdmVfY29ubmVjdGlvbikgQ29ubmVjdGlvbiBy ZW1vdmVkIGZyb20gMTI3LjAuMC4xOjQ0NjcxPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPkRldGVjdG9y IHRocmVhZDo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzIsNTk5Ojpwcm90b2NvbGRldGVjdG9y OjoyNDc6OnZkcy5NdWx0aVByb3RvY29sQWNjZXB0b3I6OihfaGFuZGxlX2Nvbm5lY3Rpb25fcmVh ZCkgRGV0ZWN0ZWQgcHJvdG9jb2wgeG1sIGZyb20gMTI3LjAuMC4xOjQ0NjcxPC9kaXY+DQo8ZGl2 IGNsYXNzPSIiPkRldGVjdG9yIHRocmVhZDo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzIsNTk5 OjpCaW5kaW5nWE1MUlBDOjoxMTczOjpYbWxEZXRlY3Rvcjo6KGhhbmRsZVNvY2tldCkgeG1sIG92 ZXIgaHR0cCBkZXRlY3RlZCBmcm9tICgnMTI3LjAuMC4xJywgNDQ2NzEpPC9kaXY+DQo8ZGl2IGNs YXNzPSIiPlRocmVhZC0xOTQ3NjQyOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozMiw2MDI6OnRh c2s6OjU5NTo6U3RvcmFnZS5UYXNrTWFuYWdlci5UYXNrOjooX3VwZGF0ZVN0YXRlKSBUYXNrPWAx ZDk5YTE2Ni1jYjlhLTQwMjUtODIxMS1hNDhlMjEwYjUyMzRgOjptb3ZpbmcgZnJvbSBzdGF0ZSBp bml0IC0mZ3Q7IHN0YXRlIHByZXBhcmluZzwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaHJlYWQtMTk0 NzY0Mjo6SU5GTzo6MjAxNS0xMS0wMyAwODo0NzozMiw2MDM6OmxvZ1V0aWxzOjo0NDo6ZGlzcGF0 Y2hlcjo6KHdyYXBwZXIpIFJ1biBhbmQgcHJvdGVjdDogcmVwb1N0YXRzKG9wdGlvbnM9Tm9uZSk8 L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhyZWFkLTE5NDc2NDI6OklORk86OjIwMTUtMTEtMDMgMDg6 NDc6MzIsNjAzOjpsb2dVdGlsczo6NDc6OmRpc3BhdGNoZXI6Oih3cmFwcGVyKSBSdW4gYW5kIHBy b3RlY3Q6IHJlcG9TdGF0cywgUmV0dXJuIHJlc3BvbnNlOiB7dSdkZTllYjczNy02OTFmLTQ2MjIt OTA3MC04OTE1MzFkNTk5YTAnOiB7J2NvZGUnOiAwLCAnYWN0dWFsJzogVHJ1ZSwgJ3ZlcnNpb24n OiAwLCAnYWNxdWlyZWQnOiBUcnVlLCAnZGVsYXknOiAnMC4wMDAzNzM2MTMnLA0KICdsYXN0Q2hl Y2snOiAnMi41JywgJ3ZhbGlkJzogVHJ1ZX0sIHUnZmU0ZmQxOWEtODcxNC00NGUwLWFlNDEtNjYz YTRiNjJkYTdhJzogeydjb2RlJzogMCwgJ2FjdHVhbCc6IFRydWUsICd2ZXJzaW9uJzogMCwgJ2Fj cXVpcmVkJzogVHJ1ZSwgJ2RlbGF5JzogJzAuMDAwNDA5NDQ2JywgJ2xhc3RDaGVjayc6ICc2LjQn LCAndmFsaWQnOiBUcnVlfSwgdSc4MjUzYTg5Yi02NTFlLTRmZjQtODY1Yi01N2FkZWYwNWQzODMn OiB7J2NvZGUnOiAwLCAnYWN0dWFsJzoNCiBUcnVlLCAndmVyc2lvbic6IDMsICdhY3F1aXJlZCc6 IFRydWUsICdkZWxheSc6ICcwLjAwMDUyMDY3MScsICdsYXN0Q2hlY2snOiAnMS44JywgJ3ZhbGlk JzogVHJ1ZX0sICdiMThlYjI5ZS04YmIxLTQ1YjktYTYwZS1hOGUwNzIxMGUwNjYnOiB7J2NvZGUn OiAwLCAnYWN0dWFsJzogVHJ1ZSwgJ3ZlcnNpb24nOiAzLCAnYWNxdWlyZWQnOiBUcnVlLCAnZGVs YXknOiAnMC4wMDA0MjQ0NDUnLCAnbGFzdENoZWNrJzogJzYuNScsICd2YWxpZCc6IFRydWV9fTwv ZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaHJlYWQtMTk0NzY0Mjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6 NDc6MzIsNjAzOjp0YXNrOjoxMTkxOjpTdG9yYWdlLlRhc2tNYW5hZ2VyLlRhc2s6OihwcmVwYXJl KSBUYXNrPWAxZDk5YTE2Ni1jYjlhLTQwMjUtODIxMS1hNDhlMjEwYjUyMzRgOjpmaW5pc2hlZDog e3UnZGU5ZWI3MzctNjkxZi00NjIyLTkwNzAtODkxNTMxZDU5OWEwJzogeydjb2RlJzogMCwgJ2Fj dHVhbCc6IFRydWUsICd2ZXJzaW9uJzogMCwgJ2FjcXVpcmVkJzoNCiBUcnVlLCAnZGVsYXknOiAn MC4wMDAzNzM2MTMnLCAnbGFzdENoZWNrJzogJzIuNScsICd2YWxpZCc6IFRydWV9LCB1J2ZlNGZk MTlhLTg3MTQtNDRlMC1hZTQxLTY2M2E0YjYyZGE3YSc6IHsnY29kZSc6IDAsICdhY3R1YWwnOiBU cnVlLCAndmVyc2lvbic6IDAsICdhY3F1aXJlZCc6IFRydWUsICdkZWxheSc6ICcwLjAwMDQwOTQ0 NicsICdsYXN0Q2hlY2snOiAnNi40JywgJ3ZhbGlkJzogVHJ1ZX0sIHUnODI1M2E4OWItNjUxZS00 ZmY0LTg2NWItNTdhZGVmMDVkMzgzJzoNCiB7J2NvZGUnOiAwLCAnYWN0dWFsJzogVHJ1ZSwgJ3Zl cnNpb24nOiAzLCAnYWNxdWlyZWQnOiBUcnVlLCAnZGVsYXknOiAnMC4wMDA1MjA2NzEnLCAnbGFz dENoZWNrJzogJzEuOCcsICd2YWxpZCc6IFRydWV9LCAnYjE4ZWIyOWUtOGJiMS00NWI5LWE2MGUt YThlMDcyMTBlMDY2Jzogeydjb2RlJzogMCwgJ2FjdHVhbCc6IFRydWUsICd2ZXJzaW9uJzogMywg J2FjcXVpcmVkJzogVHJ1ZSwgJ2RlbGF5JzogJzAuMDAwNDI0NDQ1JywgJ2xhc3RDaGVjayc6DQog JzYuNScsICd2YWxpZCc6IFRydWV9fTwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaHJlYWQtMTk0NzY0 Mjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzIsNjAzOjp0YXNrOjo1OTU6OlN0b3JhZ2UuVGFz a01hbmFnZXIuVGFzazo6KF91cGRhdGVTdGF0ZSkgVGFzaz1gMWQ5OWExNjYtY2I5YS00MDI1LTgy MTEtYTQ4ZTIxMGI1MjM0YDo6bW92aW5nIGZyb20gc3RhdGUgcHJlcGFyaW5nIC0mZ3Q7IHN0YXRl IGZpbmlzaGVkPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRocmVhZC0xOTQ3NjQyOjpERUJVRzo6MjAx NS0xMS0wMyAwODo0NzozMiw2MDQ6OnJlc291cmNlTWFuYWdlcjo6OTQwOjpTdG9yYWdlLlJlc291 cmNlTWFuYWdlci5Pd25lcjo6KHJlbGVhc2VBbGwpIE93bmVyLnJlbGVhc2VBbGwgcmVxdWVzdHMg e30gcmVzb3VyY2VzIHt9PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRocmVhZC0xOTQ3NjQyOjpERUJV Rzo6MjAxNS0xMS0wMyAwODo0NzozMiw2MDQ6OnJlc291cmNlTWFuYWdlcjo6OTc3OjpTdG9yYWdl LlJlc291cmNlTWFuYWdlci5Pd25lcjo6KGNhbmNlbEFsbCkgT3duZXIuY2FuY2VsQWxsIHJlcXVl c3RzIHt9PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRocmVhZC0xOTQ3NjQyOjpERUJVRzo6MjAxNS0x MS0wMyAwODo0NzozMiw2MDQ6OnRhc2s6Ojk5Mzo6U3RvcmFnZS5UYXNrTWFuYWdlci5UYXNrOjoo X2RlY3JlZikgVGFzaz1gMWQ5OWExNjYtY2I5YS00MDI1LTgyMTEtYTQ4ZTIxMGI1MjM0YDo6cmVm IDAgYWJvcnRpbmcgRmFsc2U8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhyZWFkLTYzNDg6OkRFQlVH OjoyMDE1LTExLTAzIDA4OjQ3OjMzLDI2MTo6bGlidmlydGNvbm5lY3Rpb246OjE0Mzo6cm9vdDo6 KHdyYXBwZXIpIFVua25vd24gbGlidmlydGVycm9yOiBlY29kZTogODAgZWRvbTogMjAgbGV2ZWw6 IDIgbWVzc2FnZTogbWV0YWRhdGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0YWRhdGEgZWxlbWVu dCBpcyBub3QgcHJlc2VudDwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaHJlYWQtNzI4Njo6REVCVUc6 OjIwMTUtMTEtMDMgMDg6NDc6MzMsNDYyOjpsaWJ2aXJ0Y29ubmVjdGlvbjo6MTQzOjpyb290Ojoo d3JhcHBlcikgVW5rbm93biBsaWJ2aXJ0ZXJyb3I6IGVjb2RlOiA4MCBlZG9tOiAyMCBsZXZlbDog MiBtZXNzYWdlOiBtZXRhZGF0YSBub3QgZm91bmQ6IFJlcXVlc3RlZCBtZXRhZGF0YSBlbGVtZW50 IGlzIG5vdCBwcmVzZW50PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPkR1bW15LTE4OTUyNjA6OkRFQlVH OjoyMDE1LTExLTAzIDA4OjQ3OjMzLDUxNDo6c3RvcmFnZV9tYWlsYm94Ojo3MzE6OlN0b3JhZ2Uu TWlzYy5leGNDbWQ6OihfY2hlY2tGb3JNYWlsKSBkZCBpZj0vcmhldi9kYXRhLWNlbnRlci8wMDAw MDAwMi0wMDAyLTAwMDItMDAwMi0wMDAwMDAwMDAzZDUvbWFzdGVyc2QvZG9tX21kL2luYm94IGlm bGFnPWRpcmVjdCxmdWxsYmxvY2sgY291bnQ9MSBicz0xMDI0MDAwIChjd2QgTm9uZSk8L2Rpdj4N CjxkaXYgY2xhc3M9IiI+RHVtbXktMTg5NTI2MDo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzMs NTQwOjpzdG9yYWdlX21haWxib3g6OjczMTo6U3RvcmFnZS5NaXNjLmV4Y0NtZDo6KF9jaGVja0Zv ck1haWwpIFNVQ0NFU1M6ICZsdDtlcnImZ3Q7ID0gJzEmIzQzOzAgcmVjb3JkcyBpblxuMSYjNDM7 MCByZWNvcmRzIG91dFxuMTAyNDAwMCBieXRlcyAoMS4wIE1CKSBjb3BpZWQsIDAuMDAzODU5MDEg cywgMjY1IE1CL3Ncbic7ICZsdDtyYyZndDsgPSAwPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRocmVh ZC03NjI3OjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozMyw5Mzg6OmxpYnZpcnRjb25uZWN0aW9u OjoxNDM6OnJvb3Q6Oih3cmFwcGVyKSBVbmtub3duIGxpYnZpcnRlcnJvcjogZWNvZGU6IDgwIGVk b206IDIwIGxldmVsOiAyIG1lc3NhZ2U6IG1ldGFkYXRhIG5vdCBmb3VuZDogUmVxdWVzdGVkIG1l dGFkYXRhIGVsZW1lbnQgaXMgbm90IHByZXNlbnQ8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhyZWFk LTc5NTE6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMzLDkzODo6bGlidmlydGNvbm5lY3Rpb246 OjE0Mzo6cm9vdDo6KHdyYXBwZXIpIFVua25vd24gbGlidmlydGVycm9yOiBlY29kZTogODAgZWRv bTogMjAgbGV2ZWw6IDIgbWVzc2FnZTogbWV0YWRhdGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0 YWRhdGEgZWxlbWVudCBpcyBub3QgcHJlc2VudDwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaHJlYWQt Mzg4Mjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzMsOTQwOjpsaWJ2aXJ0Y29ubmVjdGlvbjo6 MTQzOjpyb290Ojood3JhcHBlcikgVW5rbm93biBsaWJ2aXJ0ZXJyb3I6IGVjb2RlOiA4MCBlZG9t OiAyMCBsZXZlbDogMiBtZXNzYWdlOiBtZXRhZGF0YSBub3QgZm91bmQ6IFJlcXVlc3RlZCBtZXRh ZGF0YSBlbGVtZW50IGlzIG5vdCBwcmVzZW50PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRocmVhZC03 OTY3OjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozMyw5NDA6OmxpYnZpcnRjb25uZWN0aW9uOjox NDM6OnJvb3Q6Oih3cmFwcGVyKSBVbmtub3duIGxpYnZpcnRlcnJvcjogZWNvZGU6IDgwIGVkb206 IDIwIGxldmVsOiAyIG1lc3NhZ2U6IG1ldGFkYXRhIG5vdCBmb3VuZDogUmVxdWVzdGVkIG1ldGFk YXRhIGVsZW1lbnQgaXMgbm90IHByZXNlbnQ8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhyZWFkLTc4 OTk6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjMzLDk1MTo6bGlidmlydGNvbm5lY3Rpb246OjE0 Mzo6cm9vdDo6KHdyYXBwZXIpIFVua25vd24gbGlidmlydGVycm9yOiBlY29kZTogODAgZWRvbTog MjAgbGV2ZWw6IDIgbWVzc2FnZTogbWV0YWRhdGEgbm90IGZvdW5kOiBSZXF1ZXN0ZWQgbWV0YWRh dGEgZWxlbWVudCBpcyBub3QgcHJlc2VudDwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVs cyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzgzOjp2bWNoYW5uZWxzOjo5 Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTMzLjwvZGl2Pg0K PGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6 NDc6MzQsMzgzOjp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91 dCBvbiBmaWxlbm8gMTM1LjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5l cjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzgzOjp2bWNoYW5uZWxzOjo5Njo6dmRzOjoo X2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTM2LjwvZGl2Pg0KPGRpdiBjbGFz cz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzgz Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxl bm8gMTQ2LjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6 OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg0Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90 aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTYwLjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBD aGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg0Ojp2bWNoYW5u ZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTYxLjwv ZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEt MDMgMDg6NDc6MzQsMzg0Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykg VGltZW91dCBvbiBmaWxlbm8gMTY3LjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBM aXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg0Ojp2bWNoYW5uZWxzOjo5Njo6 dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTcxLjwvZGl2Pg0KPGRp diBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6 MzQsMzg0Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBv biBmaWxlbm8gMTcyLjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6 REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg0Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hh bmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTU5LjwvZGl2Pg0KPGRpdiBjbGFzcz0i Ij5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg1Ojp2 bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8g MTg5LjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIw MTUtMTEtMDMgMDg6NDc6MzQsMzg1Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1l b3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTkwLjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFu bmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg1Ojp2bWNoYW5uZWxz Ojo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTk1LjwvZGl2 Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMg MDg6NDc6MzQsMzg1Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGlt ZW91dCBvbiBmaWxlbm8gMTk3LjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0 ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg1Ojp2bWNoYW5uZWxzOjo5Njo6dmRz OjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMTk4LjwvZGl2Pg0KPGRpdiBj bGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQs Mzg1Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBm aWxlbm8gMjEzLjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5WTSBDaGFubmVscyBMaXN0ZW5lcjo6REVC VUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg2Ojp2bWNoYW5uZWxzOjo5Njo6dmRzOjooX2hhbmRs ZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gMjE1LjwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5W TSBDaGFubmVscyBMaXN0ZW5lcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsMzg2Ojp2bWNo YW5uZWxzOjo5Njo6dmRzOjooX2hhbmRsZV90aW1lb3V0cykgVGltZW91dCBvbiBmaWxlbm8gOTgu PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlZNIENoYW5uZWxzIExpc3RlbmVyOjpERUJVRzo6MjAxNS0x MS0wMyAwODo0NzozNCwzODY6OnZtY2hhbm5lbHM6Ojk2Ojp2ZHM6OihfaGFuZGxlX3RpbWVvdXRz KSBUaW1lb3V0IG9uIGZpbGVubyAxMjUuPC9kaXY+DQo8ZGl2IGNsYXNzPSIiPkpzb25ScGMgKFN0 b21wUmVhY3Rvcik6OkRFQlVHOjoyMDE1LTExLTAzIDA4OjQ3OjM0LDQxMjo6c3RvbXBSZWFjdG9y Ojo5ODo6QnJva2VyLlN0b21wQWRhcHRlcjo6KGhhbmRsZV9mcmFtZSkgSGFuZGxpbmcgbWVzc2Fn ZSAmbHQ7U3RvbXBGcmFtZSBjb21tYW5kPSdTRU5EJyZndDs8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+ SnNvblJwY1NlcnZlcjo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsNDEzOjpfX2luaXRfXzo6 NTMwOjpqc29ucnBjLkpzb25ScGNTZXJ2ZXI6OihzZXJ2ZV9yZXF1ZXN0cykgV2FpdGluZyBmb3Ig cmVxdWVzdDwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaHJlYWQtMTk0NzY0Mzo6REVCVUc6OjIwMTUt MTEtMDMgMDg6NDc6MzQsNDM3OjpzdG9tcFJlYWN0b3I6OjE2Mzo6eWFqc29ucnBjLlN0b21wU2Vy dmVyOjooc2VuZCkgU2VuZGluZyByZXNwb25zZTwvZGl2Pg0KPGRpdiBjbGFzcz0iIj5UaHJlYWQt NzYxMzo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzQsNzEzOjpsaWJ2aXJ0Y29ubmVjdGlvbjo6 MTQzOjpyb290Ojood3JhcHBlcikgVW5rbm93biBsaWJ2aXJ0ZXJyb3I6IGVjb2RlOiA4MCBlZG9t OiAyMCBsZXZlbDogMiBtZXNzYWdlOiBtZXRhZGF0YSBub3QgZm91bmQ6IFJlcXVlc3RlZCBtZXRh ZGF0YSBlbGVtZW50IGlzIG5vdCBwcmVzZW50PC9kaXY+DQo8ZGl2IGNsYXNzPSIiPlRocmVhZC02 MzkzOjpERUJVRzo6MjAxNS0xMS0wMyAwODo0NzozNCw3MTM6OmxpYnZpcnRjb25uZWN0aW9uOjox NDM6OnJvb3Q6Oih3cmFwcGVyKSBVbmtub3duIGxpYnZpcnRlcnJvcjogZWNvZGU6IDgwIGVkb206 IDIwIGxldmVsOiAyIG1lc3NhZ2U6IG1ldGFkYXRhIG5vdCBmb3VuZDogUmVxdWVzdGVkIG1ldGFk YXRhIGVsZW1lbnQgaXMgbm90IHByZXNlbnQ8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhyZWFkLTIy Njk0MTo6REVCVUc6OjIwMTUtMTEtMDMgMDg6NDc6MzUsNTExOjp0YXNrOjo1OTU6OlN0b3JhZ2Uu VGFza01hbmFnZXIuVGFzazo6KF91cGRhdGVTdGF0ZSkgVGFzaz1gYmRmMjY0MDEtMzI0Yy00MjIw LTkwMzQtMTljN2Q4MTZmNjQyYDo6bW92aW5nIGZyb20gc3RhdGUgaW5pdCAtJmd0OyBzdGF0ZSBw cmVwYXJpbmc8L2Rpdj4NCjxkaXYgY2xhc3M9IiI+VGhyZWFkLTQ3MDk6OkRFQlVHOjoyMDE1LTEx LTAzIDA4OjQ3OjM1LDUxMTo6dGFzazo6NTk1OjpTdG9yYWdlLlRhc2tNYW5hZ2VyLlRhc2s6Oihf dXBkYXRlU3RhdGUpIFRhc2s9YDBhYzJmODU0LTQxZTMtNDQyNy1hMGY0LTVlYWExODQyZTIxMmA6 Om1vdmluZyBmcm9tIHN0YXRlIGluaXQgLSZndDsgc3RhdGUgcHJlcGFyaW5nPC9kaXY+DQo8ZGl2 IGFwcGxlLWNvbnRlbnQtZWRpdGVkPSJ0cnVlIiBjbGFzcz0iIj4NCjxkaXYgc3R5bGU9ImxldHRl ci1zcGFjaW5nOiBub3JtYWw7IG9ycGhhbnM6IGF1dG87IHRleHQtYWxpZ246IHN0YXJ0OyB0ZXh0 LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsg d2lkb3dzOiBhdXRvOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0 aDogMHB4OyB3b3JkLXdyYXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsg LXdlYmtpdC1saW5lLWJyZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsiIGNsYXNzPSIiPg0KPHAgc3R5 bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxMHB0OyBsaW5l LWhlaWdodDogMTZweDsiIGNsYXNzPSIiPg0KPGZvbnQgY29sb3I9IiMzZDNiM2IiIGNsYXNzPSIi PjxiIGNsYXNzPSIiPi0tPC9iPjwvZm9udD48L3A+DQo8ZGl2IGNsYXNzPSIiPkNocmlzdG9waGU8 L2Rpdj4NCiZuYnNwOzxzcGFuIGNsYXNzPSJBcHBsZS1jb252ZXJ0ZWQtc3BhY2UiIHN0eWxlPSJj b2xvcjogcmdiKDAsIDAsIDApOyI+Jm5ic3A7PC9zcGFuPjwvZGl2Pg0KPC9kaXY+DQo8YnIgY2xh c3M9IiI+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1sPg0K --_000_ABC69195C9484895AAFF1900F54BBE43unilu_--
This is a multi-part message in MIME format. --------------060505020006070701060805 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Could you also attach the glusterfs and gluster mount logs from the hypervisor? (Logs would be at /var/log/glusterfs) On 11/03/2015 01:22 PM, Christophe TREFOIS wrote:
Hi,
I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.
I am running oVirt 3.5.3.
Thank you for any pointers as to where to start fixing this issue.
log excerpt --
Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125. JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing
*--*
Christophe
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--------------060505020006070701060805 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=windows-1252" http-equiv="Content-Type"> </head> <body text="#000000" bgcolor="#FFFFFF"> Could you also attach the glusterfs and gluster mount logs from the hypervisor? (Logs would be at /var/log/glusterfs)<br> <br> <div class="moz-cite-prefix">On 11/03/2015 01:22 PM, Christophe TREFOIS wrote:<br> </div> <blockquote cite="mid:ABC69195-C948-4895-AAFF-1900F54BBE43@uni.lu" type="cite"> <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> Hi, <div class=""><br class=""> </div> <div class="">I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.</div> <div class=""><br class=""> </div> <div class="">I am running oVirt 3.5.3.</div> <div class=""><br class=""> </div> <div class="">Thank you for any pointers as to where to start fixing this issue.</div> <div class=""><br class=""> </div> <div class=""> log excerpt --</div> <div class=""> <div class=""><br class=""> </div> <div class=""> <div class="">Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response</div> <div class="">Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_<a class="moz-txt-link-freetext" href="mailbox::731::Storage.Misc.excCmd::(_checkForMail)">mailbox::731::Storage.Misc.excCmd::(_checkForMail)</a> dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None)</div> <div class="">Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_<a class="moz-txt-link-freetext" href="mailbox::731::Storage.Misc.excCmd::(_checkForMail)">mailbox::731::Storage.Misc.excCmd::(_checkForMail)</a> SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0</div> <div class="">Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671</div> <div class="">Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671</div> <div class="">Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671</div> <div class="">Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671)</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing</div> <div class="">Thread-1947642::<a class="moz-txt-link-freetext" href="INFO::2015-11-03">INFO::2015-11-03</a> 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)</div> <div class="">Thread-1947642::<a class="moz-txt-link-freetext" href="INFO::2015-11-03">INFO::2015-11-03</a> 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}}</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}}</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False</div> <div class="">Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_<a class="moz-txt-link-freetext" href="mailbox::731::Storage.Misc.excCmd::(_checkForMail)">mailbox::731::Storage.Misc.excCmd::(_checkForMail)</a> dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None)</div> <div class="">Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_<a class="moz-txt-link-freetext" href="mailbox::731::Storage.Misc.excCmd::(_checkForMail)">mailbox::731::Storage.Misc.excCmd::(_checkForMail)</a> SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0</div> <div class="">Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125.</div> <div class="">JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'></div> <div class="">JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request</div> <div class="">Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response</div> <div class="">Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing</div> <div class="">Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing</div> <div apple-content-edited="true" class=""> <div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""> <p style="font-family: Arial, sans-serif; font-size: 10pt; line-height: 16px;" class=""> <font class="" color="#3d3b3b"><b class="">--</b></font></p> <div class="">Christophe</div> <span class="Apple-converted-space" style="color: rgb(0, 0, 0);"> </span></div> </div> <br class=""> </div> </div> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br> </body> </html> --------------060505020006070701060805--
--_004_E85DBDC707A44BEFA5C1F17D6176979Funilu_ Content-Type: multipart/alternative; boundary="_000_E85DBDC707A44BEFA5C1F17D6176979Funilu_" --_000_E85DBDC707A44BEFA5C1F17D6176979Funilu_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Dear Sahina, I attached the logs from the glusterfs folder. thank you for your help, Kind regards, =97 Christophe On 03 Nov 2015, at 09:04, Sahina Bose <sabose@redhat.com<mailto:sabose@redh= at.com>> wrote: Could you also attach the glusterfs and gluster mount logs from the hypervi= sor? (Logs would be at /var/log/glusterfs) On 11/03/2015 01:22 PM, Christophe TREFOIS wrote: Hi, I checked the logs on my hypervisor that contains also the overt-engine (se= lf-hosted) and I see strange unknown libvirterrors that come periodically i= n the vdsm.log file. The storage is glusterFS running on the hypervisor as = well, one NFS export domain and an ISO domain. A NFS domain from another pl= ace is in maintenance mode. I am running oVirt 3.5.3. Thank you for any pointers as to where to start fixing this issue. =97 log excerpt -- Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrp= c.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_mailbox::731::Storag= e.Misc.excCmd::(_checkForMail) dd if=3D/rhev/data-center/00000002-0002-0002= -0002-0000000003d5/mastersd/dom_md/inbox iflag=3Ddirect,fullblock count=3D1= bs=3D1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storag= e.Misc.excCmd::(_checkForMail) SUCCESS: <err> =3D '1+0 records in\n1+0 reco= rds out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> =3D= 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds= .MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:= 44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds= .MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.= 0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds= .MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml fro= m 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDe= tector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskMana= ger.Task::(_updateState) Task=3D`1d99a166-cb9a-4025-8211-a48e210b5234`::mov= ing from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(w= rapper) Run and protect: repoStats(options=3DNone) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(w= rapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-= 9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': T= rue, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a= -8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'a= cquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True},= u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'vers= ion': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'val= id': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': T= rue, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '= 6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskMan= ager.Task::(prepare) Task=3D`1d99a166-cb9a-4025-8211-a48e210b5234`::finishe= d: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'v= ersion': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', '= valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual= ': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck= ': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code':= 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671',= 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066'= : {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.0= 00424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskMana= ger.Task::(_updateState) Task=3D`1d99a166-cb9a-4025-8211-a48e210b5234`::mov= ing from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Stora= ge.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resourc= es {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Stora= ge.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskMana= ger.Task::(_decref) Task=3D`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 ab= orting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storag= e.Misc.excCmd::(_checkForMail) dd if=3D/rhev/data-center/00000002-0002-0002= -0002-0000000003d5/mastersd/dom_md/inbox iflag=3Ddirect,fullblock count=3D1= bs=3D1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storag= e.Misc.excCmd::(_checkForMail) SUCCESS: <err> =3D '1+0 records in\n1+0 reco= rds out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> =3D= 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::= (_handle_timeouts) Timeout on fileno 125. JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::B= roker.StompAdapter::(handle_frame) Handling message <StompFrame command=3D'= SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonR= pcServer::(serve_requests) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrp= c.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::= (wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metada= ta not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManag= er.Task::(_updateState) Task=3D`bdf26401-324c-4220-9034-19c7d816f642`::movi= ng from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager= .Task::(_updateState) Task=3D`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving= from state init -> state preparing -- Christophe _______________________________________________ Users mailing list Users@ovirt.org<mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users --_000_E85DBDC707A44BEFA5C1F17D6176979Funilu_ Content-Type: text/html; charset="Windows-1252" Content-ID: <E402390F86EA0D46A25C1C41BD3F0B6E@uni.lux> Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1= 252"> </head> <body> <div class=3D"" style=3D"word-wrap:break-word">Dear Sahina, <div class=3D""><br class=3D""> </div> <div class=3D"">I attached the logs from the glusterfs folder. </div> <div class=3D""><br class=3D""> </div> <div class=3D""></div> </div> <div class=3D"" style=3D"word-wrap:break-word"> <div class=3D""></div> <div class=3D""><br class=3D""> </div> <div class=3D"">thank you for your help,</div> <div class=3D""><br class=3D""> </div> <div class=3D"">Kind regards,</div> <div class=3D""><br class=3D""> </div> <div class=3D"">=97</div> <div class=3D"">Christophe <div class=3D""> <div class=3D"" style=3D"color:rgb(0,0,0); letter-spacing:normal; orphans:a= uto; text-align:start; text-indent:0px; text-transform:none; white-space:no= rmal; widows:auto; word-spacing:0px; word-wrap:break-word"> <span class=3D"x_Apple-converted-space"> </span></div> </div> <br class=3D""> <div> <blockquote type=3D"cite" class=3D""> <div class=3D"">On 03 Nov 2015, at 09:04, Sahina Bose <<a href=3D"mailto= :sabose@redhat.com" class=3D"">sabose@redhat.com</a>> wrote:</div> <br class=3D"x_Apple-interchange-newline"> <div class=3D""> <div bgcolor=3D"#FFFFFF" class=3D"">Could you also attach the glusterfs and= gluster mount logs from the hypervisor? (Logs would be at /var/log/gluster= fs)<br class=3D""> <br class=3D""> <div class=3D"x_moz-cite-prefix">On 11/03/2015 01:22 PM, Christophe TREFOIS= wrote:<br class=3D""> </div> <blockquote type=3D"cite" class=3D"">Hi, <div class=3D""><br class=3D""> </div> <div class=3D"">I checked the logs on my hypervisor that contains also the = overt-engine (self-hosted) and I see strange unknown libvirterrors that com= e periodically in the vdsm.log file. The storage is glusterFS running on th= e hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenanc= e mode.</div> <div class=3D""><br class=3D""> </div> <div class=3D"">I am running oVirt 3.5.3.</div> <div class=3D""><br class=3D""> </div> <div class=3D"">Thank you for any pointers as to where to start fixing this= issue.</div> <div class=3D""><br class=3D""> </div> <div class=3D"">=97 log excerpt --</div> <div class=3D""> <div class=3D""><br class=3D""> </div> <div class=3D""> <div class=3D"">Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReacto= r::163::yajsonrpc.StompServer::(send) Sending response</div> <div class=3D"">Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_<a c= lass=3D"x_moz-txt-link-freetext" href=3D"">mailbox::731::Storage.Misc.excCm= d::(_checkForMail)</a> dd if=3D/rhev/data-center/00000002-0002-0002-0002-00= 00000003d5/mastersd/dom_md/inbox iflag=3Ddirect,fullblock count=3D1 bs=3D1024000 (cwd None)</div> <div class=3D"">Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_<a c= lass=3D"x_moz-txt-link-freetext" href=3D"">mailbox::731::Storage.Misc.excCm= d::(_checkForMail)</a> SUCCESS: <err> =3D '1+0 records in\n1+= 0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> =3D 0</div> <div class=3D"">Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Detector thread::DEBUG::2015-11-03 08:47:32,591::protocolde= tector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection= from 127.0.0.1:44671</div> <div class=3D"">Detector thread::DEBUG::2015-11-03 08:47:32,598::protocolde= tector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection rem= oved from 127.0.0.1:44671</div> <div class=3D"">Detector thread::DEBUG::2015-11-03 08:47:32,599::protocolde= tector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected = protocol xml from 127.0.0.1:44671</div> <div class=3D"">Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXML= RPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.= 0.1', 44671)</div> <div class=3D"">Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::= Storage.TaskManager.Task::(_updateState) Task=3D`1d99a166-cb9a-4025-8211-a4= 8e210b5234`::moving from state init -> state preparing</div> <div class=3D"">Thread-1947642::<a class=3D"x_moz-txt-link-freetext" href= =3D"">INFO::2015-11-03</a> 08:47:32,603::logUtils::44::dispatcher::(wrapper= ) Run and protect: repoStats(options=3DNone)</div> <div class=3D"">Thread-1947642::<a class=3D"x_moz-txt-link-freetext" href= =3D"">INFO::2015-11-03</a> 08:47:32,603::logUtils::47::dispatcher::(wrapper= ) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-8= 91531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5'= , 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'act= ual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCh= eck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00= 0520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e0= 7210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'del= ay': '0.000424445', 'lastCheck': '6.5', 'valid': True}}</div> <div class=3D"">Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191:= :Storage.TaskManager.Task::(prepare) Task=3D`1d99a166-cb9a-4025-8211-a48e21= 0b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'a= ctual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd1= 9a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, = 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True= }, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00= 0520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e0= 7210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'del= ay': '0.000424445', 'lastCheck': '6.5', 'valid': True}}</div> <div class=3D"">Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::= Storage.TaskManager.Task::(_updateState) Task=3D`1d99a166-cb9a-4025-8211-a4= 8e210b5234`::moving from state preparing -> state finished</div> <div class=3D"">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceMan= ager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll req= uests {} resources {}</div> <div class=3D"">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceMan= ager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll reque= sts {}</div> <div class=3D"">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::= Storage.TaskManager.Task::(_decref) Task=3D`1d99a166-cb9a-4025-8211-a48e210= b5234`::ref 0 aborting False</div> <div class=3D"">Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_<a c= lass=3D"x_moz-txt-link-freetext" href=3D"">mailbox::731::Storage.Misc.excCm= d::(_checkForMail)</a> dd if=3D/rhev/data-center/00000002-0002-0002-0002-00= 00000003d5/mastersd/dom_md/inbox iflag=3Ddirect,fullblock count=3D1 bs=3D1024000 (cwd None)</div> <div class=3D"">Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_<a c= lass=3D"x_moz-txt-link-freetext" href=3D"">mailbox::731::Storage.Misc.excCm= d::(_checkForMail)</a> SUCCESS: <err> =3D '1+0 records in\n1+= 0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> =3D 0</div> <div class=3D"">Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 133.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 135.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 136.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 146.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 160.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 161.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 167.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 171.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 172.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 159.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 189.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 190.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 195.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 197.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 198.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 213.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 215.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 98.</div> <div class=3D"">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmcha= nnels::96::vds::(_handle_timeouts) Timeout on fileno 125.</div> <div class=3D"">JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::sto= mpReactor::98::Broker.StompAdapter::(handle_frame) Handling message <Sto= mpFrame command=3D'SEND'></div> <div class=3D"">JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::53= 0::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request</div> <div class=3D"">Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReacto= r::163::yajsonrpc.StompServer::(send) Sending response</div> <div class=3D"">Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnect= ion::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2= message: metadata not found: Requested metadata element is not present</di= v> <div class=3D"">Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::S= torage.TaskManager.Task::(_updateState) Task=3D`bdf26401-324c-4220-9034-19c= 7d816f642`::moving from state init -> state preparing</div> <div class=3D"">Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Sto= rage.TaskManager.Task::(_updateState) Task=3D`0ac2f854-41e3-4427-a0f4-5eaa1= 842e212`::moving from state init -> state preparing</div> <div class=3D""> <div class=3D"" style=3D"letter-spacing:normal; orphans:auto; text-align:st= art; text-indent:0px; text-transform:none; white-space:normal; widows:auto;= word-spacing:0px; word-wrap:break-word"> <p class=3D"" style=3D"font-family:Arial,sans-serif; font-size:10pt; line-h= eight:16px"> <font class=3D"" color=3D"#3d3b3b"><b class=3D"">--</b></font></p> <div class=3D"">Christophe</div> <span class=3D"x_Apple-converted-space" style=3D""> </span></div=
</div> <br class=3D""> </div> </div> <br class=3D""> <fieldset class=3D"x_mimeAttachmentHeader"></fieldset> <br class=3D""> <pre class=3D"">_______________________________________________ Users mailing list <a class=3D"x_moz-txt-link-abbreviated" href=3D"mailto:Users@ovirt.org">Use= rs@ovirt.org</a> <a class=3D"x_moz-txt-link-freetext" href=3D"http://lists.ovirt.org/mailman= /listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br class=3D""> </div> </div> </blockquote> </div> <br class=3D""> </div> </div> </body> </html> --_000_E85DBDC707A44BEFA5C1F17D6176979Funilu_-- --_004_E85DBDC707A44BEFA5C1F17D6176979Funilu_ Content-Type: application/x-gzip; name="gluster-bio2-logs.tar.gz" Content-Description: gluster-bio2-logs.tar.gz Content-Disposition: attachment; filename="gluster-bio2-logs.tar.gz"; size=4057; creation-date="Tue, 03 Nov 2015 12:44:45 GMT"; modification-date="Tue, 03 Nov 2015 12:44:45 GMT" Content-ID: <5E24E694ADC3BE449E425D5D3F4BDD8B@uni.lux> Content-Transfer-Encoding: base64 H4sIADKrOFYAA+1dXW/bOBbN8/wKYZ52gdIhL+/lh7EzwC5mMTNYzACL7ltRBCRFJUETu5s487G/ finbaZsqdmW7lihHfIlry7V0zuW5h+QVFW6uJzfzy7NjNp6a4nz5lzf/Ll8LUEJyqSXSGReCC35W 8KOe1bo93C/cXVGc3c3ni23HfenzgbZwW15cXd8v5nd/Hi0OducfSNHIfxctLgK7vEkgxLvq/vFV OfltfvP1wqE1/7Lu+CLxTyj1yH8X7fKmur+K7oaVbuGOpAB79H8tx/7fSZtV973nf6H0kn8iJERY 5n8SI/9dtDeQ1JYJwTgUHKYSphwmIvGg5Nvi5+LNh9xQstvL28UkTAUJOa1fX1zGxf37GC6Cf/e2 4B/TyLT4dV6EKze7jMX1rEi5pLq+ia+KMJ8trmcP17PLb/q+7LGt291V/G2p/SzEWWKP3c4Wj0y+ /oH56zlMHmZpjPAwvdg3RWzt/0JrKdf9XyvUmPI/pAQAY//vom3o/9aAPWb/f/KzZkpY/6zixK1e /uwvr3/8+YdpIbjlSr0t3pRXC3YXZ+421meAQkzTOxerd+rfXkZwemtaLN9LP1GcGyDpjPVMkYgM qwqZUeQZaVfGilMpjTy/vnWX8f7cUxko6shMlIKhsY55HolBaSrjZRm9EOeyTlCV5ixITwzBeeZC mY6U6cvKA3gFk9uYOsks/l785crdX3236lo316lzMX4eXLiKT9/7a/Hd932ea+vzfJ4zza3i5nPO 5CNnYX57O58lzrRRsOQs6cdFgufixv05f1hcVPO7i/L67uI+3lS1D31K5uu4WNRcrg4u5lVLpG5d HYnnC3f/7v48VOkMBRnGKw8MbeDMCkz/hOQzopQhhVPx+/Xiqnjz+sGnaL2oo2paPLn8V8U/7+6m BRPFq+J1Uox0cnz5cv5+WiBYtEqDpfTWTwnOhELx9lWxAbJl9xo8ZJNFvH3fKW7pTaG6lYfdA+nr d/vdmfEuvHt4v+VU/jZ7uPl+U4+G9G7HKrxn8GWA9V4gy4kFTbohm2OqyzfVJc6UVQp74UzqEKxN EAQFKh2rifmyNMmza8N1kNaQPDfakyZnGUibwKJAzJJyzMg0nKZQVWWkDjj7uud6EGc4SUqmTCNn ZJZrPZqgAjNUJigkBuYih1p0uI2GYwTXXZpNkAGgtYOHrGN7UuOm0yi317zZIpCOnjJbMHOAPalh NlY03HNmMHdkTw7ieyvIUoPKfVSXowagBEmZB2cOcXmgBiAB5Q7z4DWgXotpgDxqwBc1gKwx/Y6f X4QGKIGYO8yD1wCdxtyjD9hdA4xQsuOpgpeoAUZKGscCx9QAmggFZBrj2lEDtmtAwk0DQu7BmUNc HqIBNczIe14VOn0NAEFgGnOC49R3JlPf/7mKxW28v08nUXybtU73uqz8bQrB99EtYllgsbhOiBU+ Ln6PcVZsXah3s/K5A+QkXS439HYb/pmtm+awZNpW73ema10fsJkuizyNZgZE13CWuXcna1VlsIUs rSx/+yQfCTUVNjEyAasM5e1JjfTaKpGwtNIzjAGZ014zb8FoJ6SJxndgRz9AJjnoE4CsKxv/CW4a uq7U2D2QjiwRrZjZx8Z/ArMxmDvMncjxgXyvQR6KJey5z3+StuyWtNVQ0c/T1ocDNMpkM3LxGKem HbvTtRLvzXRp1JCNgz8tDdqdrFUK2EKWseZZS4hmIgWhbqwHZaV10YSSuA0sOEioOgXMoNMMBBfg opImXVBX1qaGDITheVvCNpB1awmXuBH1Wn7QKpCOLBGtmNnfEi5h1oL3uYjWPvgywPp0LGHPfb7l TEZDRZ9PW+kA5NoYkYvHODXt2J2ulXhvpkuggWxmCU9Lg3Yna5UCtpClQGPDEoKYEkwkatucU8hK 66wwNpTKsGS2BUPBK+aEQ6Yteeuc8VZ1Ufz/ETLFgfIu/m8DWYeW8BE3ErxPS9gqkI4sEa2Y2dMS PsKsuZWZw9yJHB/I98AsYc99vv1MxlMVfSZt1QfICSeS+axEnpp27E7XSrw306UUDouu4WjQ7mSt UsAWsoyo/fvnH+IU5cSgtZnf2GS4AV5WFbOoFUOsArO8UqxCZ7XWzpdcd2RtVpARIuU9sdoGsi4t 4Rq3dFJ91n+2CqRjLyS0YWZfS7iG2ZLquDRxz+DLAOvTsYQ99/n2MxlPVfS5tJUOwAnnyTOqXDzG qWnH7nStxHsLXVpCNpO6p6VBu5O1SgFbyDJGUtMS6inCxCSmRN6roM4FrbFEFpQChhwd88qk/yCA t1zrdFWhI2uzhozMCUDWpSVc4mY5l0L06FVaBdKRJaIVM/tawjXMwKnPheP2wZcB1ruDbKai3qxT PTO8yUoDuIrKKR2YqTBdfYiKedSGUXTCUEkBLe+o+68gEwJlo/sPDrIuZXONmwLoUzZbBdKRu3Ir ZvaVzTXMRjQr6DKDuRPZPJDvrSBD+u1RA3bXAEm8aTkzC84c4vJADZDK9LqryovQANQc8x4+5akB JIUdNeDoGkCUxqmZwzx4DdASm8sWowZ8UQMMN80CtcyCM4e4PFADDGDXG368PA2wUo0+YHcN0Bwg +wSVQ1wepgHJpIpmrUxmMA9IA4ayst7z0skOBWFPFqOeW/3T9fJgymQpm+WyVHtqSzC70vW4BraZ LiV4PoUQp7WUsztZq5W0LWSBEc1iS4n1txFV6uNZ+xujKxt4WTHtY8Uwiop5LktGloAHlDL6shtr s4aMuFF5P6+kDWQdWsJH3Ah1n0u+rQLp2MU3bZjZ0xI+wqyp1w132wdfBljvCTIZrk12S0TrLS6D 5CiVrlhlYjpW6/o5W8GwUnivvIgaxbGq0/MwBKcVgZ8YAr3dEDwVgGcMweoAjWhRbTAEmrTu+hFJ O8TtuY7KVhEcq3S97Y2wkVke6u1ceX3HlpBW+g62kf2653rQwDSzjpVDn9pjYGTada2VhdnYtQwH xZv3uj9+iCR6vQ3ipHRxMJM1Pbvx9gPKp+ObzUEOQsrmHl/rD6UQkHuQ5xDfh7l8IxPOuQ+mBqQl z4BsJ1pboH6eANzKwx/N3aQLuZ3/45e///fdj/tClzS4uSVvXrJckomVD8zodP4YYmDJj1lmg6/I U73wF7ubH6khU9rm/TC3NpB1PK9kk63iZPt8ukirQDq2CrZh5oCMU8MMot/dn1sHXwZY7wuy4IiZ T8dnqQFCCdFnTewL0QChTXOficxgHrwGSGvk8K1T9xqA1PMuEy9DA1BznTvMg9cAlYI58w19stQA ZQ2OY4Gja4AWMI4Fjq0BRjxTQjtqwBc1wGop+iw9fiEaYI2Wow84lgYQn4KuN76RJm8foBAq7oRn XKt6kTtaZkB75q1QpbWytNTRXoiPkFlobuAyOMg6lM0VbpansX2f22O3CqQjd+VWzOwpm2uYBee8 zymU9sGXAdZ7gWwmHLltLgn0rQHrxTUjdBlLj8wLHRgmtWLWWUjXbRVPkJSVP9aWN3lUEZ1WBLYv z3sqAM+UV6yCV5h02rpRXrH6EOqN8Doe4O4Qt+el5sEZiSyI+hEkFfp0UPpfhVOcO1cnGeygPO/r nutQyvNOLYO2L897amE2di1AAGU3da268KnPSY3T0sXBlOf17Mbbl+c9Hd9sDnIjrWjupLr6UHIg 6nNy5NQ0ahPMRJg7zAPSkibIEicWQELHQ6ldPPxxy/PUv/73079xX+iQcn+GmNMuSscTxMFphsEr 5l2JDBw6W0ZrhevosWuPkFndXPccHGTdzivVuJG0ve4E0iqQjn1neBtm9s84S5iJep1Xah98GWC9 L8gqjRHynlvOUwM0l6MGHF8DNHDZ563fL0IDDPCmtR814IsaYLSRfe4M+kI0wFgwfVZAnb4GUP1k IRJ536qToQYk3ARymbtJzSEuD9GAGmYyzQ3sM4N58BoAKezzLtPPUwMkiuwHqjnE5YEaIMn2Ws5z whoAYsqh3n5VWpN6d9YawLWzvPKOiRJFunrDEw5BsehLo6WXBqGL8rwPkCUDUD9WfeiQdSWbn+Cm JfY5tm8VSMfe5LcNM/vI5icwG9vrnQ3tgy8DrE+nCKDnPt+yCKChop8XAXw4QNt6Z/Bcdug9Ne3Y na6VeG+ky3Au0mBiQHQNR4N2J2uVAraQJTANr58mEJgin0K9fyLY5vAmK61ThEZKDiwoUTEsZWQu oGFBW4cllIIDdWFtPkJmpMp7Rr0NZJ1Zwg+4qSQqvd5K0CaQjl1f1IaZvSzhR5gJdZ8z6u2DLwOs 9wTZKKOaDzLpWwPW9VbCexls4IyXIaQ0VAJzPnpWxjIoTxArPO0NlU8rAlvesdEQgIYhWB0gJxwk fb4h5scPpULd8WT8DnF7TkZoCRJY5ROuGHXyWE5VzOoYFSkFSpgO7tj4uuc6mDs2TiyDtrxjo2Fh Nnat9OPKwIauJTj2u9Z9Wro4mMmant142wHl5+ObzUEOIIXaFOQSRHMLgMyCPIf4PsTl1zBL1eu+ 1SemJZ+BTDjlaiK1tNDP3aueykBRR2aiFAyNdczzSAxKUxkvy+iFOJdESFVyFUF6YgjOp0sv662X 05eVB/AKOvBCX/dcW5/n85yRBZ3fvfRPItbaFLNJpllZquQdhZDpG6VhqD1qqEhK6mIn5w+QKQDF 8y4PawNZhxNaa9zSQEn0ufjWKpCOLL+tmNkz1a1hJi16rcRtH3wZYL0XyGYitEpxP6a64aQ6MwEg o/qZqpE6BGtl/WwoUOlYTcyXKRJD1IbrIK0heW60J03OMpA2gUWBmCXlmJGCE4WqKiN1wNnXPdcD OZNCy+Y9L3nl2hISEkYx8i6FbySe8KKKOeG8k7IiL7C7NJsgA6Eg72fat4GsY3tS41Ynzj7zZptA OnbKbMPMAfakhlmj7XMk3j74MsB6X5BlGuWPGrC7BqSga9YqZBacOcTlgRqAoJuPj8oM5sFrAEky mU9TZKkBSljd560VL0QDlMReK95fhAYYEKMP2F0DkEvq+tGQL1ADkJNq3mSZGczD1gA7QaNtc0/A UQO2a0DCzRJin7dZvwQNqGFWlvqs8nkJGqBQCdHPcsU49f3lqe+hVGb1vKz8SWUWbq/MerpQ/0xl 1noVTy4fY51JteipLU/vTNe6PmALXVZKOyS6hrPMvTNZ6yqDzWQhr3eo+OZsbNvb3VX8bRWSIdES 79htIufy5qFm+PUPzF/PYfIwu57cPEwv4h/v53cL9kecTZL6t/8NnprifPmXN/9yofSZACXqlXhE OOMAiPqs4Me77I8tXam7K4qzu/l8se24L30+0PZp7+EwlTDlkHqPofVGBetQqO5Ldnt5u6gljYSc 1q8vLuPi/n0MF8G/q4Xtw6HT4td5Ea7cLCnk9axISbi6vomvijCfJRfwkBRv7JdjG9vYxja2sfXZ /g9EsT3IABgBAA== --_004_E85DBDC707A44BEFA5C1F17D6176979Funilu_--
This is a multi-part message in MIME format. --------------030807040906060707010807 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Ok, so the mount logs do not have any errors - only messages regarding dir creation and file renames. Yaniv's already answered regarding the log messages. On 11/03/2015 06:14 PM, Christophe TREFOIS wrote:
Dear Sahina,
I attached the logs from the glusterfs folder.
thank you for your help,
Kind regards,
Christophe
On 03 Nov 2015, at 09:04, Sahina Bose <sabose@redhat.com <mailto:sabose@redhat.com>> wrote:
Could you also attach the glusterfs and gluster mount logs from the hypervisor? (Logs would be at /var/log/glusterfs)
On 11/03/2015 01:22 PM, Christophe TREFOIS wrote:
Hi,
I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.
I am running oVirt 3.5.3.
Thank you for any pointers as to where to start fixing this issue.
log excerpt --
Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125. JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing
*--*
Christophe
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
--------------030807040906060707010807 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=windows-1252" http-equiv="Content-Type"> </head> <body text="#000000" bgcolor="#FFFFFF"> Ok, so the mount logs do not have any errors - only messages regarding dir creation and file renames.<br> <br> Yaniv's already answered regarding the log messages.<br> <br> <div class="moz-cite-prefix">On 11/03/2015 06:14 PM, Christophe TREFOIS wrote:<br> </div> <blockquote cite="mid:E85DBDC7-07A4-4BEF-A5C1-F17D6176979F@uni.lu" type="cite"> <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> <div class="" style="word-wrap:break-word">Dear Sahina, <div class=""><br class=""> </div> <div class="">I attached the logs from the glusterfs folder. </div> <div class=""><br class=""> </div> </div> <div class="" style="word-wrap:break-word"> <div class=""><br class=""> </div> <div class="">thank you for your help,</div> <div class=""><br class=""> </div> <div class="">Kind regards,</div> <div class=""><br class=""> </div> <div class=""></div> <div class="">Christophe <div class=""> <div class="" style="color:rgb(0,0,0); letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; word-wrap:break-word"> <span class="x_Apple-converted-space"> </span></div> </div> <br class=""> <div> <blockquote type="cite" class=""> <div class="">On 03 Nov 2015, at 09:04, Sahina Bose <<a moz-do-not-send="true" href="mailto:sabose@redhat.com" class=""><a class="moz-txt-link-abbreviated" href="mailto:sabose@redhat.com">sabose@redhat.com</a></a>> wrote:</div> <br class="x_Apple-interchange-newline"> <div class=""> <div bgcolor="#FFFFFF" class="">Could you also attach the glusterfs and gluster mount logs from the hypervisor? (Logs would be at /var/log/glusterfs)<br class=""> <br class=""> <div class="x_moz-cite-prefix">On 11/03/2015 01:22 PM, Christophe TREFOIS wrote:<br class=""> </div> <blockquote type="cite" class="">Hi, <div class=""><br class=""> </div> <div class="">I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.</div> <div class=""><br class=""> </div> <div class="">I am running oVirt 3.5.3.</div> <div class=""><br class=""> </div> <div class="">Thank you for any pointers as to where to start fixing this issue.</div> <div class=""><br class=""> </div> <div class=""> log excerpt --</div> <div class=""> <div class=""><br class=""> </div> <div class=""> <div class="">Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response</div> <div class="">Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_<a moz-do-not-send="true" class="x_moz-txt-link-freetext" href=""><a class="moz-txt-link-freetext" href="mailbox::731::Storage.Misc.excCmd::(_checkForMail)">mailbox::731::Storage.Misc.excCmd::(_checkForMail)</a></a> dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None)</div> <div class="">Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_<a moz-do-not-send="true" class="x_moz-txt-link-freetext" href=""><a class="moz-txt-link-freetext" href="mailbox::731::Storage.Misc.excCmd::(_checkForMail)">mailbox::731::Storage.Misc.excCmd::(_checkForMail)</a></a> SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0</div> <div class="">Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671</div> <div class="">Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671</div> <div class="">Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671</div> <div class="">Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671)</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing</div> <div class="">Thread-1947642::<a moz-do-not-send="true" class="x_moz-txt-link-freetext" href=""><a class="moz-txt-link-freetext" href="INFO::2015-11-03">INFO::2015-11-03</a></a> 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)</div> <div class="">Thread-1947642::<a moz-do-not-send="true" class="x_moz-txt-link-freetext" href=""><a class="moz-txt-link-freetext" href="INFO::2015-11-03">INFO::2015-11-03</a></a> 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}}</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}}</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div> <div class="">Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False</div> <div class="">Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_<a moz-do-not-send="true" class="x_moz-txt-link-freetext" href=""><a class="moz-txt-link-freetext" href="mailbox::731::Storage.Misc.excCmd::(_checkForMail)">mailbox::731::Storage.Misc.excCmd::(_checkForMail)</a></a> dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None)</div> <div class="">Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_<a moz-do-not-send="true" class="x_moz-txt-link-freetext" href=""><a class="moz-txt-link-freetext" href="mailbox::731::Storage.Misc.excCmd::(_checkForMail)">mailbox::731::Storage.Misc.excCmd::(_checkForMail)</a></a> SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0</div> <div class="">Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98.</div> <div class="">VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125.</div> <div class="">JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'></div> <div class="">JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request</div> <div class="">Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response</div> <div class="">Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present</div> <div class="">Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing</div> <div class="">Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing</div> <div class=""> <div class="" style="letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; word-wrap:break-word"> <p class="" style="font-family:Arial,sans-serif; font-size:10pt; line-height:16px"> <font class="" color="#3d3b3b"><b class="">--</b></font></p> <div class="">Christophe</div> <span class="x_Apple-converted-space" style=""> </span></div> </div> <br class=""> </div> </div> <br class=""> <fieldset class="x_mimeAttachmentHeader"></fieldset> <br class=""> <pre class="">_______________________________________________ Users mailing list <a moz-do-not-send="true" class="x_moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a moz-do-not-send="true" class="x_moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br class=""> </div> </div> </blockquote> </div> <br class=""> </div> </div> </blockquote> <br> </body> </html> --------------030807040906060707010807--
On Tue, Nov 3, 2015 at 9:52 AM, Christophe TREFOIS < christophe.trefois@uni.lu> wrote:
Hi,
I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.
I am running oVirt 3.5.3.
Thank you for any pointers as to where to start fixing this issue.
— log excerpt --
Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
This (depending on your host OS version, 6.x or 7.x) is either https://bugzilla.redhat.com/show_bug.cgi?id=1220474 or https://bugzilla.redhat.com/show_bug.cgi?id=1260864 Y. Dummy-1895260::DEBUG::2015-11-03
08:47:31,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125.
We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related. Y.
JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing
*--* Christophe
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/11/15 09:30, Yaniv Kaul wrote:
This (depending on your host OS version, 6.x or 7.x) is either https://bugzilla.redhat.com/show_bug.cgi?id=1220474 or https://bugzilla.redhat.com/show_bug.cgi?id=1260864 Y.
Those bugs are marked as private and thus can't be displayed by mere mortals. - -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +495772 293100 F: +495772 293333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhaus en Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJWOLmsAAoJEMby9TMDAbQRbeYP/0mOi74azrEPhp46juxpMZc3 vzZ7ciftMtsln86rKTuQGBUcJ0Sqb0RBWZ8bsDex9+Ngx9rTiwpDwfzWxV5zmraa 9SFnsw3pz1nL/C2swTQQEfN2uq54U6WEvPlyW0dlDwrlKsVRtD7qQka81aspZHA/ PPCej64mRSKUnP4c0s8sDfzNyzEdJW75x67kksP7FNZSuYDRO6qk3ciLbRV21vBi A6Rp54Tedw/2y+zra7KqkRzP4DwiL55k0/5FAxssSw0hoeRzjHCf4Y6wiScXBtRK Naocwp+LL/0LgvtVfbS5CGTGvPGmpqCjb4uLWWXIIhnAaZ3AB4XNFKctOKqE3DMj qp+m5+vgzhNqyQbdQuiGIP64nesuI3z16ov03pjqxZGZvszyLCT8+cRAzI5vhBIb 5CiWors+az9buraLOfXiuZNHrawpvP+h/q3o/a974c3CvJ2U8ZiPIhg21cxpziP4 Xi9z1PoAbghoEwceVas5+t04rPHIqgL2k7SowyU3wDFD8cfWTd0b1fics5ryoR7g CEjJIT3hRmc2/TT2wZoxmp60WYwTH3n2bRKiCdZM3V0lO/3XOL6EL3qPYp4qX8Xl k3yj5zqugkuM5efXlr5I7NIQkNQD1KWvB/QjhWzFaSq2HJ9C2XiHV003DmgoPfkH GkwO4mp9RF1f126JHNCG =QtoB -----END PGP SIGNATURE-----
On Tue, Nov 3, 2015 at 3:42 PM, Sven Kieske <s.kieske@mittwald.de> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/11/15 09:30, Yaniv Kaul wrote:
This (depending on your host OS version, 6.x or 7.x) is either https://bugzilla.redhat.com/show_bug.cgi?id=1220474 or
*Bug 1220474* <https://bugzilla.redhat.com/show_bug.cgi?id=1220474> - libvirt: Error 'metadata not found: Requested metadata element is not present' flooding journal Fixed in libvirt 1.2.16, for RHEL 7.2.
*Bug 1260864* <https://bugzilla.redhat.com/show_bug.cgi?id=1260864> - libvirt: Error 'metadata not found: Requested metadata element is not present' flooding journal Fixed in libvirt 1.2.16, for RHEL 6.8.
Those bugs are marked as private and thus can't be displayed by mere mortals.
My apologies. Upstream fix: commit ecc997fd43b1b1987dadb45fb0ef987d56638476 Author: Peter Krempa <pkrempa@redhat.com> Date: Tue May 12 13:55:30 2015 +0200 daemon: Suppress logging of VIR_ERR_NO_DOMAIN_METADATA Y.
- -- Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +495772 293100 F: +495772 293333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhaus en Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAEBAgAGBQJWOLmsAAoJEMby9TMDAbQRbeYP/0mOi74azrEPhp46juxpMZc3 vzZ7ciftMtsln86rKTuQGBUcJ0Sqb0RBWZ8bsDex9+Ngx9rTiwpDwfzWxV5zmraa 9SFnsw3pz1nL/C2swTQQEfN2uq54U6WEvPlyW0dlDwrlKsVRtD7qQka81aspZHA/ PPCej64mRSKUnP4c0s8sDfzNyzEdJW75x67kksP7FNZSuYDRO6qk3ciLbRV21vBi A6Rp54Tedw/2y+zra7KqkRzP4DwiL55k0/5FAxssSw0hoeRzjHCf4Y6wiScXBtRK Naocwp+LL/0LgvtVfbS5CGTGvPGmpqCjb4uLWWXIIhnAaZ3AB4XNFKctOKqE3DMj qp+m5+vgzhNqyQbdQuiGIP64nesuI3z16ov03pjqxZGZvszyLCT8+cRAzI5vhBIb 5CiWors+az9buraLOfXiuZNHrawpvP+h/q3o/a974c3CvJ2U8ZiPIhg21cxpziP4 Xi9z1PoAbghoEwceVas5+t04rPHIqgL2k7SowyU3wDFD8cfWTd0b1fics5ryoR7g CEjJIT3hRmc2/TT2wZoxmp60WYwTH3n2bRKiCdZM3V0lO/3XOL6EL3qPYp4qX8Xl k3yj5zqugkuM5efXlr5I7NIQkNQD1KWvB/QjhWzFaSq2HJ9C2XiHV003DmgoPfkH GkwO4mp9RF1f126JHNCG =QtoB -----END PGP SIGNATURE----- _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Tue, Nov 3, 2015 at 10:30 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Tue, Nov 3, 2015 at 9:52 AM, Christophe TREFOIS <christophe.trefois@uni.lu> wrote:
Hi,
I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.
I am running oVirt 3.5.3.
Thank you for any pointers as to where to start fixing this issue.
— log excerpt --
Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
This (depending on your host OS version, 6.x or 7.x) is either https://bugzilla.redhat.com/show_bug.cgi?id=1220474 or https://bugzilla.redhat.com/show_bug.cgi?id=1260864 Y.
The error about missing metadata is just noise in the log, nothing to worry about. Adding Martin
Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125.
We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related. Y.
Adding Vinzenz
JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing
--
Christophe
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Dear Nir, Thank you for your help. I shall ignore the messages then :) Kind regards, — Christophe Dr Christophe Trefois, Dipl.-Ing. Technical Specialist / Post-Doc UNIVERSITÉ DU LUXEMBOURG LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE Campus Belval | House of Biomedicine 6, avenue du Swing L-4367 Belvaux T: +352 46 66 44 6124 F: +352 46 66 44 6949 http://www.uni.lu/lcsb ---- This message is confidential and may contain privileged information. It is intended for the named recipient only. If you receive it in error please notify me and permanently delete the original message and any copies. ----
On 06 Nov 2015, at 12:23, Nir Soffer <nsoffer@redhat.com> wrote:
On Tue, Nov 3, 2015 at 10:30 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Tue, Nov 3, 2015 at 9:52 AM, Christophe TREFOIS <christophe.trefois@uni.lu> wrote:
Hi,
I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.
I am running oVirt 3.5.3.
Thank you for any pointers as to where to start fixing this issue.
— log excerpt --
Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
This (depending on your host OS version, 6.x or 7.x) is either https://bugzilla.redhat.com/show_bug.cgi?id=1220474 or https://bugzilla.redhat.com/show_bug.cgi?id=1260864 Y.
The error about missing metadata is just noise in the log, nothing to worry about.
Adding Martin
Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125.
We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related. Y.
Adding Vinzenz
JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing
--
Christophe
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi,
Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
we fixed this on vdsm side of oVirt 3.6 too: https://gerrit.ovirt.org/#/c/45799/ But Nir is correct. This was just noise, it went away when any QoS was defined for the VM and was not important otherwise.
VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125. We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related.
No relation to the metadata log messages. Can't help here. Best regards -- Martin Sivak SLA / oVirt On Fri, Nov 6, 2015 at 1:14 PM, Christophe TREFOIS <christophe.trefois@uni.lu> wrote:
Dear Nir,
Thank you for your help. I shall ignore the messages then :)
Kind regards,
— Christophe
Dr Christophe Trefois, Dipl.-Ing. Technical Specialist / Post-Doc
UNIVERSITÉ DU LUXEMBOURG
LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE Campus Belval | House of Biomedicine 6, avenue du Swing L-4367 Belvaux T: +352 46 66 44 6124 F: +352 46 66 44 6949 http://www.uni.lu/lcsb
---- This message is confidential and may contain privileged information. It is intended for the named recipient only. If you receive it in error please notify me and permanently delete the original message and any copies. ----
On 06 Nov 2015, at 12:23, Nir Soffer <nsoffer@redhat.com> wrote:
On Tue, Nov 3, 2015 at 10:30 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Tue, Nov 3, 2015 at 9:52 AM, Christophe TREFOIS <christophe.trefois@uni.lu> wrote:
Hi,
I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.
I am running oVirt 3.5.3.
Thank you for any pointers as to where to start fixing this issue.
— log excerpt --
Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
This (depending on your host OS version, 6.x or 7.x) is either https://bugzilla.redhat.com/show_bug.cgi?id=1220474 or https://bugzilla.redhat.com/show_bug.cgi?id=1260864 Y.
The error about missing metadata is just noise in the log, nothing to worry about.
Adding Martin
Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125.
We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related. Y.
Adding Vinzenz
JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing
--
Christophe
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On 06 Nov 2015, at 13:35, Martin Sivak <msivak@redhat.com> wrote:
Hi,
Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
we fixed this on vdsm side of oVirt 3.6 too:
https://gerrit.ovirt.org/#/c/45799/
But Nir is correct. This was just noise, it went away when any QoS was defined for the VM and was not important otherwise.
VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125. We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related.
nope unless there is a ridiculous number of them they are about ovirt agent connectivity (which breaks around time of start/stop/migration, etc) but normally should not happen, when you run ovirt-guest-agent inside guest When you don’t you’ll see it every so often per each VM
No relation to the metadata log messages. Can't help here.
Best regards
-- Martin Sivak SLA / oVirt
On Fri, Nov 6, 2015 at 1:14 PM, Christophe TREFOIS <christophe.trefois@uni.lu> wrote:
Dear Nir,
Thank you for your help. I shall ignore the messages then :)
Kind regards,
— Christophe
Dr Christophe Trefois, Dipl.-Ing. Technical Specialist / Post-Doc
UNIVERSITÉ DU LUXEMBOURG
LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE Campus Belval | House of Biomedicine 6, avenue du Swing L-4367 Belvaux T: +352 46 66 44 6124 F: +352 46 66 44 6949 http://www.uni.lu/lcsb
---- This message is confidential and may contain privileged information. It is intended for the named recipient only. If you receive it in error please notify me and permanently delete the original message and any copies. ----
On 06 Nov 2015, at 12:23, Nir Soffer <nsoffer@redhat.com> wrote:
On Tue, Nov 3, 2015 at 10:30 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Tue, Nov 3, 2015 at 9:52 AM, Christophe TREFOIS <christophe.trefois@uni.lu> wrote:
Hi,
I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance mode.
I am running oVirt 3.5.3.
Thank you for any pointers as to where to start fixing this issue.
— log excerpt --
Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
This (depending on your host OS version, 6.x or 7.x) is either https://bugzilla.redhat.com/show_bug.cgi?id=1220474 or https://bugzilla.redhat.com/show_bug.cgi?id=1260864 Y.
The error about missing metadata is just noise in the log, nothing to worry about.
Adding Martin
Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Detector thread::DEBUG::2015-11-03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-0000000003d5/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125.
We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related. Y.
Adding Vinzenz
JsonRpc (StompReactor)::DEBUG::2015-11-03 08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing
--
Christophe
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
HI michal, Thanks for the tips. We're now deploying the agents on all VMs (around 60) and will see for similar entries in the logs. Have a nice evening, -- Christophe
-----Original Message----- From: Michal Skrivanek [mailto:mskrivan@redhat.com] Sent: vendredi 6 novembre 2015 14:33 To: Christophe TREFOIS <christophe.trefois@uni.lu> Cc: users@ovirt.org Subject: Re: [ovirt-users] Unknown libvirterror - where to start?
We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related.
On 06 Nov 2015, at 13:35, Martin Sivak <msivak@redhat.com> wrote:
Hi,
Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
we fixed this on vdsm side of oVirt 3.6 too:
https://gerrit.ovirt.org/#/c/45799/
But Nir is correct. This was just noise, it went away when any QoS was defined for the VM and was not important otherwise.
VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno
nope unless there is a ridiculous number of them they are about ovirt agent connectivity (which breaks around time of start/stop/migration, etc) but normally should not happen, when you run ovirt-guest-agent inside guest When you don’t you’ll see it every so often per each VM
No relation to the metadata log messages. Can't help here.
Best regards
-- Martin Sivak SLA / oVirt
On Fri, Nov 6, 2015 at 1:14 PM, Christophe TREFOIS <christophe.trefois@uni.lu> wrote:
Dear Nir,
Thank you for your help. I shall ignore the messages then :)
Kind regards,
— Christophe
Dr Christophe Trefois, Dipl.-Ing. Technical Specialist / Post-Doc
UNIVERSITÉ DU LUXEMBOURG
LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE Campus Belval |
Biomedicine 6, avenue du Swing L-4367 Belvaux T: +352 46 66 44 6124 F: +352 46 66 44 6949 http://www.uni.lu/lcsb
---- This message is confidential and may contain privileged information. It is intended for the named recipient only. If you receive it in error please notify me and permanently delete the original message and any copies. ----
On 06 Nov 2015, at 12:23, Nir Soffer <nsoffer@redhat.com> wrote:
On Tue, Nov 3, 2015 at 10:30 AM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Tue, Nov 3, 2015 at 9:52 AM, Christophe TREFOIS <christophe.trefois@uni.lu> wrote:
Hi,
I checked the logs on my hypervisor that contains also the overt-engine (self-hosted) and I see strange unknown libvirterrors that come periodically in the vdsm.log file. The storage is glusterFS running on the hypervisor as well, one NFS export domain and an ISO domain. A NFS domain from another place is in maintenance
mode.
I am running oVirt 3.5.3.
Thank you for any pointers as to where to start fixing this issue.
— log excerpt --
Thread-1947641::DEBUG::2015-11-03 08:47:31,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-8108::DEBUG::2015-11-03 08:47:31,410::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present
This (depending on your host OS version, 6.x or 7.x) is either https://bugzilla.redhat.com/show_bug.cgi?id=1220474 or https://bugzilla.redhat.com/show_bug.cgi?id=1260864 Y.
The error about missing metadata is just noise in the log, nothing to worry about.
Adding Martin
Dummy-1895260::DEBUG::2015-11-03 08:47:31,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkFo rMail) dd if=/rhev/data-center/00000002-0002-0002-0002-
0000000003d5/mastersd
/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:31,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkFo rMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00331278 s, 309 MB/s\n'; <rc> = 0 Thread-7913::DEBUG::2015-11-03 08:47:32,298::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-5682::DEBUG::2015-11-03 08:47:32,417::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Detector thread::DEBUG::2015-11- 03 08:47:32,591::protocoldetector::187::vds.MultiProtocolAcceptor::(_ add_connection) Adding connection from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,598::protocoldetector::201::vds.MultiProtocolAcceptor::(_ remove_connection) Connection removed from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::protocoldetector::247::vds.MultiProtocolAcceptor::(_ handle_connection_read) Detected protocol xml from 127.0.0.1:44671 Detector thread::DEBUG::2015-11-03 08:47:32,599::BindingXMLRPC::1173::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 44671) Thread-1947642::DEBUG::2015-11-03 08:47:32,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state init -> state preparing Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) Thread-1947642::INFO::2015-11-03 08:47:32,603::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'de9eb737-691f-4622-9070- 891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b-57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e- a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::finished: {u'de9eb737-691f-4622-9070-891531d599a0': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000373613', 'lastCheck': '2.5', 'valid': True}, u'fe4fd19a-8714-44e0-ae41-663a4b62da7a': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000409446', 'lastCheck': '6.4', 'valid': True}, u'8253a89b-651e-4ff4-865b- 57adef05d383': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000520671', 'lastCheck': '1.8', 'valid': True}, 'b18eb29e-8bb1-45b9-a60e-a8e07210e066': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000424445', 'lastCheck': '6.5', 'valid': True}} Thread-1947642::DEBUG::2015-11-03 08:47:32,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::moving from state preparing -> state finished Thread-1947642::DEBUG::2015-11-03
08:47:32,604::resourceManager::940::Storage.ResourceManager.Owner:
:(releaseAll) Owner.releaseAll requests {} resources {} Thread-1947642::DEBUG::2015-11-03
08:47:32,604::resourceManager::977::Storage.ResourceManager.Owner:
:(cancelAll) Owner.cancelAll requests {} Thread-1947642::DEBUG::2015-11-03 08:47:32,604::task::993::Storage.TaskManager.Task::(_decref) Task=`1d99a166-cb9a-4025-8211-a48e210b5234`::ref 0 aborting False Thread-6348::DEBUG::2015-11-03 08:47:33,261::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7286::DEBUG::2015-11-03 08:47:33,462::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Dummy-1895260::DEBUG::2015-11-03 08:47:33,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkFo rMail) dd if=/rhev/data-center/00000002-0002-0002-0002- 0000000003d5/mastersd /dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) Dummy-1895260::DEBUG::2015-11-03 08:47:33,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkFo rMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00385901 s, 265 MB/s\n'; <rc> = 0 Thread-7627::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7951::DEBUG::2015-11-03 08:47:33,938::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-3882::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7967::DEBUG::2015-11-03 08:47:33,940::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-7899::DEBUG::2015-11-03 08:47:33,951::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 133. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 135. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 136. VM Channels Listener::DEBUG::2015-11-03 08:47:34,383::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 146. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 160. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 161. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 167. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 171. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 172. VM Channels Listener::DEBUG::2015-11-03 08:47:34,384::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 159. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 189. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 190. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 195. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 197. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 198. VM Channels Listener::DEBUG::2015-11-03 08:47:34,385::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 213. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 215. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 98. VM Channels Listener::DEBUG::2015-11-03 08:47:34,386::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 125.
We've seen this as well. I don't think there's a specific bug filed on this issue. I wonder if they related. Y.
Adding Vinzenz
JsonRpc (StompReactor)::DEBUG::2015-11-03
08:47:34,412::stompReactor::98::Broker.StompAdapter::(handle_frame
) Handling message <StompFrame command='SEND'> JsonRpcServer::DEBUG::2015-11-03 08:47:34,413::__init__::530::jsonrpc.JsonRpcServer::(serve_request s) Waiting for request Thread-1947643::DEBUG::2015-11-03 08:47:34,437::stompReactor::163::yajsonrpc.StompServer::(send) Sending response Thread-7613::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-6393::DEBUG::2015-11-03 08:47:34,713::libvirtconnection::143::root::(wrapper) Unknown
House of libvirterror: libvirterror: libvirterror: libvirterror: libvirterror: libvirterror: libvirterror: libvirterror: libvirterror: libvirterror: libvirterror: libvirterror:
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata element is not present Thread-226941::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdf26401-324c-4220-9034-19c7d816f642`::moving from state init -> state preparing Thread-4709::DEBUG::2015-11-03 08:47:35,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac2f854-41e3-4427-a0f4-5eaa1842e212`::moving from state init -> state preparing
--
Christophe
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (7)
-
Christophe TREFOIS -
Martin Sivak -
Michal Skrivanek -
Nir Soffer -
Sahina Bose -
Sven Kieske -
Yaniv Kaul