From fawzy_ibrhim at yahoo.com Mon Sep 1 16:47:09 2014 Content-Type: multipart/mixed; boundary="===============4130061032256644009==" MIME-Version: 1.0 From: Fawzy Ibrhim To: users at ovirt.org Subject: [ovirt-users] Volume XXX contains a apparently corrupt brick(s). Date: Mon, 01 Sep 2014 13:44:18 -0700 Message-ID: <1409604258.73116.YahooMailNeo@web124503.mail.ne1.yahoo.com> --===============4130061032256644009== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable ---1103591728-816431412-1409604258=3D:73116 Content-Type: text/plain; charset=3Dus-ascii Dear All; I have GlusterFS storage domain in my oVirt Engine Version: 3.4.3-1.el6. One of the peers get disconnected due to network cable unplugged and after = that I failed to add/remove disks from my GlusterFS domain. I have healed the GlusterFS volume and removed the corrupted index file and= healed the GlusterFS volume. As workaround, I have stopped all the VM and put the GlusterFS domian in ma= intenance mode and activated it again and now I can add/remove disks from t= he GlusterFS domain. But I still have the below warnings in my ovirt-engine.log:- 2014-09-01 23:09:47,550 WARN [org.ovirt.engine.core.bll.gluster.GlusterSyn= cJob] (DefaultQuartzScheduler_Worker-26) Volume XXX contains a apparently c= orrupt brick(s). Hence will not add it to engine at this point. 2014-09-01 23:09:52,558 INFO [org.ovirt.engine.core.vdsbroker.gluster.Glus= terVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-61) START, Gluster= VolumesListVDSCommand(HostName =3D host1, HostId =3D 0de0e7d7-a3dd-4776-939= 3-c36b8c4b452d), log id: 2b6381b0 2014-09-01 23:09:52,588 WARN [org.ovirt.engine.core.vdsbroker.gluster.Glus= terVolumesListReturnForXmlRpc] (DefaultQuartzScheduler_Worker-61) Could not= add brick host2:/var/lib/exports/gluster/brick1 to volume fca9d8a4-392d-4e= 55-a2ac-8e09a75c5cd2 - server uuid d7763283-10b6-4880-a7b3-48d0353205b0 not= found in cluster 00000001-0001-0001-0001-00000000033a 2014-09-01 23:09:52,594 INFO [org.ovirt.engine.core.vdsbroker.gluster.Glus= terVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, Gluste= rVolumesListVDSCommand, return: {fca9d8a4-392d-4e55-a2ac-8e09a75c5cd2=3Dorg= .ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity(a)10= 7254ac}, log id: 2b6381b0 Best Regards Fawzy Ibrahim ---1103591728-816431412-1409604258=3D:73116 Content-Type: text/html; charset=3Dus-ascii
Dear All;

<= /div>
I have GlusterFS storage domain in my oVirt Engin= e Version: 3.4.3-1.el6.

One of the peers get disconnected due to network cable unplugged and = after that I failed to add/remove disks from my GlusterFS domain.

I have healed the GlusterFS volume and remo= ved the corrupted index file and healed the GlusterFS volume.

As workaround, I have stopped a= ll the VM and put the GlusterFS domian in maintenance mode and activated it= again and now I can add/remove disks from the GlusterFS domain.

But I still have the below = warnings in my ovirt-engine.log:-

2014-09-01 = 23:09:47,550 WARN  [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] = (DefaultQuartzScheduler_Worker-26) Volume XXX contains a apparently corrupt= brick(s). Hence will not add it to engine at this point.
2014-09-= 01 23:09:52,558 INFO  [org.ovirt.engine.core.vdsbroker.gluster.Gluster= VolumesListVDSCommand] (DefaultQuartzScheduler_Worker-61) START, GlusterVol= umesListVDSCommand(HostName =3D host1, HostId =3D 0de0e7d7-a3dd-4776-9393-c= 36b8c4b452d), log id: 2b6381b0
2014-09-01 23:09:52= ,588 WARN  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesList= ReturnForXmlRpc] (DefaultQuartzScheduler_Worker-61) Could not add brick hos= t2:/var/lib/exports/gluster/brick1 to volume fca9d8a4-392d-4e55-a2ac-8e09a7= 5c5cd2 - server uuid d7763283-10b6-4880-a7b3-48d0353205b0 not found in clus= ter 00000001-0001-0001-0001-00000000033a
2014-09-0= 1 23:09:52,594 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterV= olumesListVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, GlusterVol= umesListVDSCommand, return: {fca9d8a4-392d-4e55-a2ac-8e09a75c5cd2=3Dorg.ovirt.engine.core.common.busin= essentities.gluster.GlusterVolumeEntity(a)107254ac}, log id: 2b6381b0
=

Best Regards
Fawzy Ibrahim
---1103591728-816431412-1409604258=3D:73116-- --===============4130061032256644009== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" LS0tMTEwMzU5MTcyOC04MTY0MzE0MTItMTQwOTYwNDI1OD06NzMxMTYKQ29udGVudC1UeXBlOiB0 ZXh0L3BsYWluOyBjaGFyc2V0PXVzLWFzY2lpCgpEZWFyIEFsbDsKCkkgaGF2ZSBHbHVzdGVyRlMg c3RvcmFnZSBkb21haW4gaW4gbXkgb1ZpcnQgRW5naW5lIFZlcnNpb246IDMuNC4zLTEuZWw2LgoK T25lIG9mIHRoZSBwZWVycyBnZXQgZGlzY29ubmVjdGVkIGR1ZSB0byBuZXR3b3JrIGNhYmxlIHVu cGx1Z2dlZCBhbmQgYWZ0ZXIgdGhhdCBJIGZhaWxlZCB0byBhZGQvcmVtb3ZlIGRpc2tzIGZyb20g bXkgR2x1c3RlckZTIGRvbWFpbi4KCkkgaGF2ZSBoZWFsZWQgdGhlIEdsdXN0ZXJGUyB2b2x1bWUg YW5kIHJlbW92ZWQgdGhlIGNvcnJ1cHRlZCBpbmRleCBmaWxlIGFuZCBoZWFsZWQgdGhlIEdsdXN0 ZXJGUyB2b2x1bWUuCgpBcyB3b3JrYXJvdW5kLCBJIGhhdmUgc3RvcHBlZCBhbGwgdGhlIFZNIGFu ZCBwdXQgdGhlIEdsdXN0ZXJGUyBkb21pYW4gaW4gbWFpbnRlbmFuY2UgbW9kZSBhbmQgYWN0aXZh dGVkIGl0IGFnYWluIGFuZCBub3cgSSBjYW4gYWRkL3JlbW92ZSBkaXNrcyBmcm9tIHRoZSBHbHVz dGVyRlMgZG9tYWluLgoKQnV0IEkgc3RpbGwgaGF2ZSB0aGUgYmVsb3cgd2FybmluZ3MgaW4gbXkg b3ZpcnQtZW5naW5lLmxvZzotCgoKMjAxNC0wOS0wMSAyMzowOTo0Nyw1NTAgV0FSTiAgW29yZy5v dmlydC5lbmdpbmUuY29yZS5ibGwuZ2x1c3Rlci5HbHVzdGVyU3luY0pvYl0gKERlZmF1bHRRdWFy dHpTY2hlZHVsZXJfV29ya2VyLTI2KSBWb2x1bWUgWFhYIGNvbnRhaW5zIGEgYXBwYXJlbnRseSBj b3JydXB0IGJyaWNrKHMpLiBIZW5jZSB3aWxsIG5vdCBhZGQgaXQgdG8gZW5naW5lIGF0IHRoaXMg cG9pbnQuCjIwMTQtMDktMDEgMjM6MDk6NTIsNTU4IElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNv cmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0VkRTQ29tbWFuZF0gKERlZmF1 bHRRdWFydHpTY2hlZHVsZXJfV29ya2VyLTYxKSBTVEFSVCwgR2x1c3RlclZvbHVtZXNMaXN0VkRT Q29tbWFuZChIb3N0TmFtZSA9IGhvc3QxLCBIb3N0SWQgPSAwZGUwZTdkNy1hM2RkLTQ3NzYtOTM5 My1jMzZiOGM0YjQ1MmQpLCBsb2cgaWQ6IDJiNjM4MWIwCjIwMTQtMDktMDEgMjM6MDk6NTIsNTg4 IFdBUk4gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZv bHVtZXNMaXN0UmV0dXJuRm9yWG1sUnBjXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXIt NjEpIENvdWxkIG5vdCBhZGQgYnJpY2sgaG9zdDI6L3Zhci9saWIvZXhwb3J0cy9nbHVzdGVyL2Jy aWNrMSB0byB2b2x1bWUgZmNhOWQ4YTQtMzkyZC00ZTU1LWEyYWMtOGUwOWE3NWM1Y2QyIC0gc2Vy dmVyIHV1aWQgZDc3NjMyODMtMTBiNi00ODgwLWE3YjMtNDhkMDM1MzIwNWIwIG5vdCBmb3VuZCBp biBjbHVzdGVyIDAwMDAwMDAxLTAwMDEtMDAwMS0wMDAxLTAwMDAwMDAwMDMzYQoyMDE0LTA5LTAx IDIzOjA5OjUyLDU5NCBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci5nbHVz dGVyLkdsdXN0ZXJWb2x1bWVzTGlzdFZEU0NvbW1hbmRdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVy X1dvcmtlci02MSkgRklOSVNILCBHbHVzdGVyVm9sdW1lc0xpc3RWRFNDb21tYW5kLCByZXR1cm46 IHtmY2E5ZDhhNC0zOTJkLTRlNTUtYTJhYy04ZTA5YTc1YzVjZDI9b3JnLm92aXJ0LmVuZ2luZS5j b3JlLmNvbW1vbi5idXNpbmVzc2VudGl0aWVzLmdsdXN0ZXIuR2x1c3RlclZvbHVtZUVudGl0eUAx MDcyNTRhY30sIGxvZyBpZDogMmI2MzgxYjAKCkJlc3QgUmVnYXJkcwpGYXd6eSBJYnJhaGltCgot LS0xMTAzNTkxNzI4LTgxNjQzMTQxMi0xNDA5NjA0MjU4PTo3MzExNgpDb250ZW50LVR5cGU6IHRl eHQvaHRtbDsgY2hhcnNldD11cy1hc2NpaQoKPGh0bWw+PGJvZHk+PGRpdiBzdHlsZT0iY29sb3I6 IzAwMDsgYmFja2dyb3VuZC1jb2xvcjojZmZmOyBmb250LWZhbWlseTp0aW1lcyBuZXcgcm9tYW4s IG5ldyB5b3JrLCB0aW1lcywgc2VyaWY7Zm9udC1zaXplOjhwdCI+PGRpdiBzdHlsZT0iIiBjbGFz cz0iIj5EZWFyIEFsbDs8L2Rpdj48ZGl2IHN0eWxlPSIiIGNsYXNzPSIiPjxiciBzdHlsZT0iIiBj bGFzcz0iIj48L2Rpdj48ZGl2IHN0eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LXNpemU6 IDEwLjY2NjdweDsgZm9udC1mYW1pbHk6IHRpbWVzIG5ldyByb21hbixuZXcgeW9yayx0aW1lcyxz ZXJpZjsgYmFja2dyb3VuZC1jb2xvcjogdHJhbnNwYXJlbnQ7IGZvbnQtc3R5bGU6IG5vcm1hbDsi IGNsYXNzPSIiPkkgaGF2ZSBHbHVzdGVyRlMgc3RvcmFnZSBkb21haW4gaW4gbXkgb1ZpcnQgRW5n aW5lIFZlcnNpb246IDMuNC4zLTEuZWw2LjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwg MCwgMCk7IGZvbnQtc2l6ZTogMTAuNjY2N3B4OyBmb250LWZhbWlseTogdGltZXMgbmV3IHJvbWFu LG5ldyB5b3JrLHRpbWVzLHNlcmlmOyBiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVudDsgZm9u dC1zdHlsZTogbm9ybWFsOyIgY2xhc3M9IiI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9yOiBy Z2IoMCwgMCwgMCk7IGZvbnQtc2l6ZTogMTAuNjY2N3B4OyBmb250LWZhbWlseTogdGltZXMgbmV3 IHJvbWFuLG5ldyB5b3JrLHRpbWVzLHNlcmlmOyBiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVu dDsgZm9udC1zdHlsZTogbm9ybWFsOyIgY2xhc3M9IiI+T25lIG9mIHRoZSBwZWVycyBnZXQgZGlz Y29ubmVjdGVkIGR1ZSB0byBuZXR3b3JrIGNhYmxlIHVucGx1Z2dlZCBhbmQgYWZ0ZXIgdGhhdCBJ IGZhaWxlZCB0byBhZGQvcmVtb3ZlIGRpc2tzIGZyb20gbXkgR2x1c3RlckZTIGRvbWFpbi48L2Rp dj48ZGl2IHN0eWxlPSJjb2xvcjoKIHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAxMC42NjY3cHg7 IGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMsc2VyaWY7IGJhY2tn cm91bmQtY29sb3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3JtYWw7IiBjbGFzcz0iIj48 YnI+PC9kaXY+PGRpdiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAxMC42 NjY3cHg7IGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMsc2VyaWY7 IGJhY2tncm91bmQtY29sb3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3JtYWw7IiBjbGFz cz0iIj5JIGhhdmUgaGVhbGVkIHRoZSBHbHVzdGVyRlMgdm9sdW1lIGFuZCByZW1vdmVkIHRoZSBj b3JydXB0ZWQgaW5kZXggZmlsZSBhbmQgaGVhbGVkIHRoZSBHbHVzdGVyRlMgdm9sdW1lLjwvZGl2 PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtc2l6ZTogMTAuNjY2N3B4OyBm b250LWZhbWlseTogdGltZXMgbmV3IHJvbWFuLG5ldyB5b3JrLHRpbWVzLHNlcmlmOyBiYWNrZ3Jv dW5kLWNvbG9yOiB0cmFuc3BhcmVudDsgZm9udC1zdHlsZTogbm9ybWFsOyIgY2xhc3M9IiI+PGJy PjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtc2l6ZTogMTAuNjY2 N3B4OyBmb250LWZhbWlseTogdGltZXMgbmV3IHJvbWFuLG5ldyB5b3JrLHRpbWVzLHNlcmlmOyBi YWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVudDsgZm9udC1zdHlsZTogbm9ybWFsOyIgY2xhc3M9 IiI+QXMgd29ya2Fyb3VuZCwgSSBoYXZlIHN0b3BwZWQgYWxsIHRoZSBWTSBhbmQgcHV0IHRoZSBH bHVzdGVyRlMgZG9taWFuIGluIG1haW50ZW5hbmNlIG1vZGUgYW5kIGFjdGl2YXRlZCBpdCBhZ2Fp biBhbmQgbm93IEkgY2FuIGFkZC9yZW1vdmUgZGlza3MgZnJvbSB0aGUgR2x1c3RlckZTIGRvbWFp bi48L2Rpdj48ZGl2CiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAxMC42 NjY3cHg7IGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMsc2VyaWY7 IGJhY2tncm91bmQtY29sb3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3JtYWw7IiBjbGFz cz0iIj48YnI+PC9kaXY+PGRpdiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXpl OiAxMC42NjY3cHg7IGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMs c2VyaWY7IGJhY2tncm91bmQtY29sb3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3JtYWw7 IiBjbGFzcz0iIj5CdXQgSSBzdGlsbCBoYXZlIHRoZSBiZWxvdyB3YXJuaW5ncyBpbiBteSBvdmly dC1lbmdpbmUubG9nOi08YnIgY2xhc3M9IiIgc3R5bGU9IiI+PC9kaXY+PGRpdiBjbGFzcz0iIiBz dHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAxMC42NjY3cHg7IGZvbnQtZmFt aWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMsc2VyaWY7IGJhY2tncm91bmQtY29s b3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3JtYWw7Ij48YnIgc3R5bGU9IiIgY2xhc3M9 IiI+PC9kaXY+PGRpdiBjbGFzcz0iIiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1z aXplOiAxMC42NjY3cHg7IGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGlt ZXMsc2VyaWY7IGJhY2tncm91bmQtY29sb3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3Jt YWw7Ij4yMDE0LTA5LTAxIDIzOjA5OjQ3LDU1MCBXQVJOJm5ic3A7IFtvcmcub3ZpcnQuZW5naW5l LmNvcmUuYmxsLmdsdXN0ZXIuR2x1c3RlclN5bmNKb2JdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVy X1dvcmtlci0yNikgVm9sdW1lIFhYWCBjb250YWlucyBhIGFwcGFyZW50bHkgY29ycnVwdCBicmlj ayhzKS4gSGVuY2UKIHdpbGwgbm90IGFkZCBpdCB0byBlbmdpbmUgYXQgdGhpcyBwb2ludC48YnIg c3R5bGU9IiIgY2xhc3M9IiI+MjAxNC0wOS0wMSAyMzowOTo1Miw1NTggSU5GTyZuYnNwOyBbb3Jn Lm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci5nbHVzdGVyLkdsdXN0ZXJWb2x1bWVzTGlzdFZE U0NvbW1hbmRdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci02MSkgU1RBUlQsIEdsdXN0 ZXJWb2x1bWVzTGlzdFZEU0NvbW1hbmQoSG9zdE5hbWUgPSBob3N0MSwgSG9zdElkID0gMGRlMGU3 ZDctYTNkZC00Nzc2LTkzOTMtYzM2YjhjNGI0NTJkKSwgbG9nIGlkOiAyYjYzODFiMDxiciBzdHls ZT0iIiBjbGFzcz0iIj4yMDE0LTA5LTAxIDIzOjA5OjUyLDU4OCBXQVJOJm5ic3A7IFtvcmcub3Zp cnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJu Rm9yWG1sUnBjXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNjEpIENvdWxkIG5vdCBh ZGQgYnJpY2sgaG9zdDI6L3Zhci9saWIvZXhwb3J0cy9nbHVzdGVyL2JyaWNrMSB0byB2b2x1bWUg ZmNhOWQ4YTQtMzkyZC00ZTU1LWEyYWMtOGUwOWE3NWM1Y2QyIC0gc2VydmVyIHV1aWQgZDc3NjMy ODMtMTBiNi00ODgwLWE3YjMtNDhkMDM1MzIwNWIwIG5vdCBmb3VuZCBpbiBjbHVzdGVyIDAwMDAw MDAxLTAwMDEtMDAwMS0wMDAxLTAwMDAwMDAwMDMzYTxiciBzdHlsZT0iIiBjbGFzcz0iIj4yMDE0 LTA5LTAxIDIzOjA5OjUyLDU5NCBJTkZPJm5ic3A7IFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRz YnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0VkRTQ29tbWFuZF0gKERlZmF1bHRRdWFy dHpTY2hlZHVsZXJfV29ya2VyLTYxKSBGSU5JU0gsIEdsdXN0ZXJWb2x1bWVzTGlzdFZEU0NvbW1h bmQsIHJldHVybjoKIHtmY2E5ZDhhNC0zOTJkLTRlNTUtYTJhYy04ZTA5YTc1YzVjZDI9b3JnLm92 aXJ0LmVuZ2luZS5jb3JlLmNvbW1vbi5idXNpbmVzc2VudGl0aWVzLmdsdXN0ZXIuR2x1c3RlclZv bHVtZUVudGl0eUAxMDcyNTRhY30sIGxvZyBpZDogMmI2MzgxYjA8L2Rpdj48ZGl2IGNsYXNzPSIi IHN0eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LXNpemU6IDEwLjY2NjdweDsgZm9udC1m YW1pbHk6IHRpbWVzIG5ldyByb21hbixuZXcgeW9yayx0aW1lcyxzZXJpZjsgYmFja2dyb3VuZC1j b2xvcjogdHJhbnNwYXJlbnQ7IGZvbnQtc3R5bGU6IG5vcm1hbDsiPjxicj48L2Rpdj48ZGl2IGNs YXNzPSIiIHN0eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LXNpemU6IDEwLjY2NjdweDsg Zm9udC1mYW1pbHk6IHRpbWVzIG5ldyByb21hbixuZXcgeW9yayx0aW1lcyxzZXJpZjsgYmFja2dy b3VuZC1jb2xvcjogdHJhbnNwYXJlbnQ7IGZvbnQtc3R5bGU6IG5vcm1hbDsiPkJlc3QgUmVnYXJk czwvZGl2PjxkaXYgY2xhc3M9IiIgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtc2l6 ZTogMTAuNjY2N3B4OyBmb250LWZhbWlseTogdGltZXMgbmV3IHJvbWFuLG5ldyB5b3JrLHRpbWVz LHNlcmlmOyBiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVudDsgZm9udC1zdHlsZTogbm9ybWFs OyI+RmF3enkgSWJyYWhpbTxicj48L2Rpdj48L2Rpdj48L2JvZHk+PC9odG1sPgotLS0xMTAzNTkx NzI4LTgxNjQzMTQxMi0xNDA5NjA0MjU4PTo3MzExNi0tCg== --===============4130061032256644009==-- From sabose at redhat.com Tue Sep 2 02:50:22 2014 Content-Type: multipart/mixed; boundary="===============0659353303169605013==" MIME-Version: 1.0 From: Sahina Bose To: users at ovirt.org Subject: Re: [ovirt-users] Volume XXX contains a apparently corrupt brick(s). Date: Tue, 02 Sep 2014 12:20:18 +0530 Message-ID: <540568AA.2060402@redhat.com> In-Reply-To: 1409604258.73116.YahooMailNeo@web124503.mail.ne1.yahoo.com --===============0659353303169605013== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------030707050409040804000705 Content-Type: text/plain; charset=3DISO-8859-1; format=3Dflowed Content-Transfer-Encoding: 7bit On 09/02/2014 02:14 AM, Fawzy Ibrhim wrote: > Dear All; > > I have GlusterFS storage domain in my oVirt Engine Version: 3.4.3-1.el6. > > One of the peers get disconnected due to network cable unplugged and = > after that I failed to add/remove disks from my GlusterFS domain. > > I have healed the GlusterFS volume and removed the corrupted index = > file and healed the GlusterFS volume. > > As workaround, I have stopped all the VM and put the GlusterFS domian = > in maintenance mode and activated it again and now I can add/remove = > disks from the GlusterFS domain. > > But I still have the below warnings in my ovirt-engine.log:- > > 2014-09-01 23:09:47,550 WARN = > [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] = > (DefaultQuartzScheduler_Worker-26) Volume XXX contains a apparently = > corrupt brick(s). Hence will not add it to engine at this point. > 2014-09-01 23:09:52,558 INFO = > [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] = > (DefaultQuartzScheduler_Worker-61) START, = > GlusterVolumesListVDSCommand(HostName =3D host1, HostId =3D = > 0de0e7d7-a3dd-4776-9393-c36b8c4b452d), log id: 2b6381b0 > 2014-09-01 23:09:52,588 WARN = > [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRp= c] = > (DefaultQuartzScheduler_Worker-61) Could not add brick = > host2:/var/lib/exports/gluster/brick1 to volume = > fca9d8a4-392d-4e55-a2ac-8e09a75c5cd2 - server uuid = > d7763283-10b6-4880-a7b3-48d0353205b0 not found in cluster = > 00000001-0001-0001-0001-00000000033a > 2014-09-01 23:09:52,594 INFO = > [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] = > (DefaultQuartzScheduler_Worker-61) FINISH, = > GlusterVolumesListVDSCommand, return: = > {fca9d8a4-392d-4e55-a2ac-8e09a75c5cd2=3Dorg.ovirt.engine.core.common.busi= nessentities.gluster.GlusterVolumeEntity(a)107254ac}, = > log id: 2b6381b0 > It looks like you're managing your gluster storage using oVirt as well. = Since one of your hosts got disconnected (host2 ?), did the host entry = get removed from the list of hosts in your cluster? The gluster volume has a brick on one of the removed hosts, and it is = unable to synchronize the brick information returned from gluster cli to = the engine's database. > Best Regards > Fawzy Ibrahim > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users --------------030707050409040804000705 Content-Type: text/html; charset=3DISO-8859-1 Content-Transfer-Encoding: 7bit
On 09/02/2014 02:14 AM, Fawzy Ibrhim wrote:
Dear All;

I have GlusterFS storage domain in my oVirt Engine Version: 3.4.3-1.el6.
One of the peers get disconnected due to network cable unplugged and after that I failed to add/remove disks from my GlusterFS domain.
I have healed the GlusterFS volume and removed the corrupted index file and healed the GlusterFS volume.
As workaround, I have stopped all the VM and put the GlusterFS domian in maintenance mode and activated it again and now I can add/remove disks from the GlusterFS domain.
But I still have the below warnings in my ovirt-engine.log:-

2014-09-01 23:09:47,550 WARN  [org.ovirt.engine.core.bll.gluster.GlusterSyncJob] (DefaultQuartzScheduler_Worker-26) Volume XXX contains a apparently corrupt brick(s). Hence will not add it to engine at this point.
2014-09-01 23:09:52,558 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCom= mand] (DefaultQuartzScheduler_Worker-61) START, GlusterVolumesListVDSCommand(HostName =3D host1, HostId =3D 0de0e7d7-a3dd-4776-9393-c36b8c4b452d), log id: 2b6381b0
2014-09-01 23:09:52,588 WARN  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn= ForXmlRpc] (DefaultQuartzScheduler_Worker-61) Could not add brick host2:/var/lib/exports/gluster/brick1 to volume fca9d8a4-392d-4e55-a2ac-8e09a75c5cd2 - server uuid d7763283-10b6-4880-a7b3-48d0353205b0 not found in cluster 00000001-0001-0001-0001-00000000033a
2014-09-01 23:09:52,594 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCom= mand] (DefaultQuartzScheduler_Worker-61) FINISH, GlusterVolumesListVDSCommand, return: {fca9d8a4-392d-4e55-a2ac-8e09a75c5cd2=3Dorg.ovirt.engine.core.com= mon.businessentities.gluster.GlusterVolumeEntity(a)107254ac}, log id: 2b6381b0



It looks like you're managing your gluster storage using oVirt as well. Since one of your hosts got disconnected (host2 ?), did the host entry get removed from the list of hosts in your cluster?

The gluster volume has a brick on one of the removed hosts, and it is unable to synchronize the brick information returned from gluster cli to the engine's database.



Best Regards
Fawzy Ibrahim


_______________________________________________
Users mailing list
Use=
rs(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

--------------030707050409040804000705-- --===============0659353303169605013== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wMzA3MDcwNTA0MDkwNDA4MDQwMDA3MDUKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PUlTTy04ODU5LTE7IGZvcm1hdD1mbG93ZWQKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzog N2JpdAoKCk9uIDA5LzAyLzIwMTQgMDI6MTQgQU0sIEZhd3p5IElicmhpbSB3cm90ZToKPiBEZWFy IEFsbDsKPgo+IEkgaGF2ZSBHbHVzdGVyRlMgc3RvcmFnZSBkb21haW4gaW4gbXkgb1ZpcnQgRW5n aW5lIFZlcnNpb246IDMuNC4zLTEuZWw2Lgo+Cj4gT25lIG9mIHRoZSBwZWVycyBnZXQgZGlzY29u bmVjdGVkIGR1ZSB0byBuZXR3b3JrIGNhYmxlIHVucGx1Z2dlZCBhbmQgCj4gYWZ0ZXIgdGhhdCBJ IGZhaWxlZCB0byBhZGQvcmVtb3ZlIGRpc2tzIGZyb20gbXkgR2x1c3RlckZTIGRvbWFpbi4KPgo+ IEkgaGF2ZSBoZWFsZWQgdGhlIEdsdXN0ZXJGUyB2b2x1bWUgYW5kIHJlbW92ZWQgdGhlIGNvcnJ1 cHRlZCBpbmRleCAKPiBmaWxlIGFuZCBoZWFsZWQgdGhlIEdsdXN0ZXJGUyB2b2x1bWUuCj4KPiBB cyB3b3JrYXJvdW5kLCBJIGhhdmUgc3RvcHBlZCBhbGwgdGhlIFZNIGFuZCBwdXQgdGhlIEdsdXN0 ZXJGUyBkb21pYW4gCj4gaW4gbWFpbnRlbmFuY2UgbW9kZSBhbmQgYWN0aXZhdGVkIGl0IGFnYWlu IGFuZCBub3cgSSBjYW4gYWRkL3JlbW92ZSAKPiBkaXNrcyBmcm9tIHRoZSBHbHVzdGVyRlMgZG9t YWluLgo+Cj4gQnV0IEkgc3RpbGwgaGF2ZSB0aGUgYmVsb3cgd2FybmluZ3MgaW4gbXkgb3ZpcnQt ZW5naW5lLmxvZzotCj4KPiAyMDE0LTA5LTAxIDIzOjA5OjQ3LDU1MCBXQVJOIAo+IFtvcmcub3Zp cnQuZW5naW5lLmNvcmUuYmxsLmdsdXN0ZXIuR2x1c3RlclN5bmNKb2JdIAo+IChEZWZhdWx0UXVh cnR6U2NoZWR1bGVyX1dvcmtlci0yNikgVm9sdW1lIFhYWCBjb250YWlucyBhIGFwcGFyZW50bHkg Cj4gY29ycnVwdCBicmljayhzKS4gSGVuY2Ugd2lsbCBub3QgYWRkIGl0IHRvIGVuZ2luZSBhdCB0 aGlzIHBvaW50Lgo+IDIwMTQtMDktMDEgMjM6MDk6NTIsNTU4IElORk8gCj4gW29yZy5vdmlydC5l bmdpbmUuY29yZS52ZHNicm9rZXIuZ2x1c3Rlci5HbHVzdGVyVm9sdW1lc0xpc3RWRFNDb21tYW5k XSAKPiAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNjEpIFNUQVJULCAKPiBHbHVzdGVy Vm9sdW1lc0xpc3RWRFNDb21tYW5kKEhvc3ROYW1lID0gaG9zdDEsIEhvc3RJZCA9IAo+IDBkZTBl N2Q3LWEzZGQtNDc3Ni05MzkzLWMzNmI4YzRiNDUyZCksIGxvZyBpZDogMmI2MzgxYjAKPiAyMDE0 LTA5LTAxIDIzOjA5OjUyLDU4OCBXQVJOIAo+IFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJv a2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJuRm9yWG1sUnBjXSAKPiAoRGVmYXVs dFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNjEpIENvdWxkIG5vdCBhZGQgYnJpY2sgCj4gaG9zdDI6 L3Zhci9saWIvZXhwb3J0cy9nbHVzdGVyL2JyaWNrMSB0byB2b2x1bWUgCj4gZmNhOWQ4YTQtMzky ZC00ZTU1LWEyYWMtOGUwOWE3NWM1Y2QyIC0gc2VydmVyIHV1aWQgCj4gZDc3NjMyODMtMTBiNi00 ODgwLWE3YjMtNDhkMDM1MzIwNWIwIG5vdCBmb3VuZCBpbiBjbHVzdGVyIAo+IDAwMDAwMDAxLTAw MDEtMDAwMS0wMDAxLTAwMDAwMDAwMDMzYQo+IDIwMTQtMDktMDEgMjM6MDk6NTIsNTk0IElORk8g Cj4gW29yZy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuZ2x1c3Rlci5HbHVzdGVyVm9sdW1l c0xpc3RWRFNDb21tYW5kXSAKPiAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNjEpIEZJ TklTSCwgCj4gR2x1c3RlclZvbHVtZXNMaXN0VkRTQ29tbWFuZCwgcmV0dXJuOiAKPiB7ZmNhOWQ4 YTQtMzkyZC00ZTU1LWEyYWMtOGUwOWE3NWM1Y2QyPW9yZy5vdmlydC5lbmdpbmUuY29yZS5jb21t b24uYnVzaW5lc3NlbnRpdGllcy5nbHVzdGVyLkdsdXN0ZXJWb2x1bWVFbnRpdHlAMTA3MjU0YWN9 LCAKPiBsb2cgaWQ6IDJiNjM4MWIwCj4KCgpJdCBsb29rcyBsaWtlIHlvdSdyZSBtYW5hZ2luZyB5 b3VyIGdsdXN0ZXIgc3RvcmFnZSB1c2luZyBvVmlydCBhcyB3ZWxsLiAKU2luY2Ugb25lIG9mIHlv dXIgaG9zdHMgZ290IGRpc2Nvbm5lY3RlZCAoaG9zdDIgPyksIGRpZCB0aGUgaG9zdCBlbnRyeSAK Z2V0IHJlbW92ZWQgZnJvbSB0aGUgbGlzdCBvZiBob3N0cyBpbiB5b3VyIGNsdXN0ZXI/CgpUaGUg Z2x1c3RlciB2b2x1bWUgaGFzIGEgYnJpY2sgb24gb25lIG9mIHRoZSByZW1vdmVkIGhvc3RzLCBh bmQgaXQgaXMgCnVuYWJsZSB0byBzeW5jaHJvbml6ZSB0aGUgYnJpY2sgaW5mb3JtYXRpb24gcmV0 dXJuZWQgZnJvbSBnbHVzdGVyIGNsaSB0byAKdGhlIGVuZ2luZSdzIGRhdGFiYXNlLgoKCgo+IEJl c3QgUmVnYXJkcwo+IEZhd3p5IElicmFoaW0KPgo+Cj4gX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KPiBVc2VycyBtYWlsaW5nIGxpc3QKPiBVc2Vyc0Bvdmly dC5vcmcKPiBodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMKCgot LS0tLS0tLS0tLS0tLTAzMDcwNzA1MDQwOTA0MDgwNDAwMDcwNQpDb250ZW50LVR5cGU6IHRleHQv aHRtbDsgY2hhcnNldD1JU08tODg1OS0xCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQK CjxodG1sPgogIDxoZWFkPgogICAgPG1ldGEgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PUlT Ty04ODU5LTEiCiAgICAgIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSI+CiAgPC9oZWFkPgogIDxi b2R5IGJnY29sb3I9IiNGRkZGRkYiIHRleHQ9IiMwMDAwMDAiPgogICAgPGJyPgogICAgPGRpdiBj bGFzcz0ibW96LWNpdGUtcHJlZml4Ij5PbiAwOS8wMi8yMDE0IDAyOjE0IEFNLCBGYXd6eSBJYnJo aW0KICAgICAgd3JvdGU6PGJyPgogICAgPC9kaXY+CiAgICA8YmxvY2txdW90ZQogICAgICBjaXRl PSJtaWQ6MTQwOTYwNDI1OC43MzExNi5ZYWhvb01haWxOZW9Ad2ViMTI0NTAzLm1haWwubmUxLnlh aG9vLmNvbSIKICAgICAgdHlwZT0iY2l0ZSI+CiAgICAgIDxkaXYgc3R5bGU9ImNvbG9yOiMwMDA7 IGJhY2tncm91bmQtY29sb3I6I2ZmZjsgZm9udC1mYW1pbHk6dGltZXMKICAgICAgICBuZXcgcm9t YW4sIG5ldyB5b3JrLCB0aW1lcywgc2VyaWY7Zm9udC1zaXplOjhwdCI+CiAgICAgICAgPGRpdiBz dHlsZT0iIiBjbGFzcz0iIj5EZWFyIEFsbDs8L2Rpdj4KICAgICAgICA8ZGl2IHN0eWxlPSIiIGNs YXNzPSIiPjxiciBzdHlsZT0iIiBjbGFzcz0iIj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8ZGl2 IHN0eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LXNpemU6IDEwLjY2NjdweDsKICAgICAg ICAgIGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMsc2VyaWY7CiAg ICAgICAgICBiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVudDsgZm9udC1zdHlsZTogbm9ybWFs OyIgY2xhc3M9IiI+SQogICAgICAgICAgaGF2ZSBHbHVzdGVyRlMgc3RvcmFnZSBkb21haW4gaW4g bXkgb1ZpcnQgRW5naW5lIFZlcnNpb246CiAgICAgICAgICAzLjQuMy0xLmVsNi48L2Rpdj4KICAg ICAgICA8ZGl2IHN0eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LXNpemU6IDEwLjY2Njdw eDsKICAgICAgICAgIGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMs c2VyaWY7CiAgICAgICAgICBiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVudDsgZm9udC1zdHls ZTogbm9ybWFsOyIgY2xhc3M9IiI+PGJyPgogICAgICAgIDwvZGl2PgogICAgICAgIDxkaXYgc3R5 bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtc2l6ZTogMTAuNjY2N3B4OwogICAgICAgICAg Zm9udC1mYW1pbHk6IHRpbWVzIG5ldyByb21hbixuZXcgeW9yayx0aW1lcyxzZXJpZjsKICAgICAg ICAgIGJhY2tncm91bmQtY29sb3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3JtYWw7IiBj bGFzcz0iIj5PbmUKICAgICAgICAgIG9mIHRoZSBwZWVycyBnZXQgZGlzY29ubmVjdGVkIGR1ZSB0 byBuZXR3b3JrIGNhYmxlIHVucGx1Z2dlZAogICAgICAgICAgYW5kIGFmdGVyIHRoYXQgSSBmYWls ZWQgdG8gYWRkL3JlbW92ZSBkaXNrcyBmcm9tIG15IEdsdXN0ZXJGUwogICAgICAgICAgZG9tYWlu LjwvZGl2PgogICAgICAgIDxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtc2l6 ZTogMTAuNjY2N3B4OwogICAgICAgICAgZm9udC1mYW1pbHk6IHRpbWVzIG5ldyByb21hbixuZXcg eW9yayx0aW1lcyxzZXJpZjsKICAgICAgICAgIGJhY2tncm91bmQtY29sb3I6IHRyYW5zcGFyZW50 OyBmb250LXN0eWxlOiBub3JtYWw7IiBjbGFzcz0iIj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAg ICAgPGRpdiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAxMC42NjY3cHg7 CiAgICAgICAgICBmb250LWZhbWlseTogdGltZXMgbmV3IHJvbWFuLG5ldyB5b3JrLHRpbWVzLHNl cmlmOwogICAgICAgICAgYmFja2dyb3VuZC1jb2xvcjogdHJhbnNwYXJlbnQ7IGZvbnQtc3R5bGU6 IG5vcm1hbDsiIGNsYXNzPSIiPkkKICAgICAgICAgIGhhdmUgaGVhbGVkIHRoZSBHbHVzdGVyRlMg dm9sdW1lIGFuZCByZW1vdmVkIHRoZSBjb3JydXB0ZWQKICAgICAgICAgIGluZGV4IGZpbGUgYW5k IGhlYWxlZCB0aGUgR2x1c3RlckZTIHZvbHVtZS48L2Rpdj4KICAgICAgICA8ZGl2IHN0eWxlPSJj b2xvcjogcmdiKDAsIDAsIDApOyBmb250LXNpemU6IDEwLjY2NjdweDsKICAgICAgICAgIGZvbnQt ZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMsc2VyaWY7CiAgICAgICAgICBi YWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVudDsgZm9udC1zdHlsZTogbm9ybWFsOyIgY2xhc3M9 IiI+PGJyPgogICAgICAgIDwvZGl2PgogICAgICAgIDxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwg MCwgMCk7IGZvbnQtc2l6ZTogMTAuNjY2N3B4OwogICAgICAgICAgZm9udC1mYW1pbHk6IHRpbWVz IG5ldyByb21hbixuZXcgeW9yayx0aW1lcyxzZXJpZjsKICAgICAgICAgIGJhY2tncm91bmQtY29s b3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3JtYWw7IiBjbGFzcz0iIj5BcwogICAgICAg ICAgd29ya2Fyb3VuZCwgSSBoYXZlIHN0b3BwZWQgYWxsIHRoZSBWTSBhbmQgcHV0IHRoZSBHbHVz dGVyRlMKICAgICAgICAgIGRvbWlhbiBpbiBtYWludGVuYW5jZSBtb2RlIGFuZCBhY3RpdmF0ZWQg aXQgYWdhaW4gYW5kIG5vdyBJCiAgICAgICAgICBjYW4gYWRkL3JlbW92ZSBkaXNrcyBmcm9tIHRo ZSBHbHVzdGVyRlMgZG9tYWluLjwvZGl2PgogICAgICAgIDxkaXYgc3R5bGU9ImNvbG9yOiByZ2Io MCwgMCwgMCk7IGZvbnQtc2l6ZTogMTAuNjY2N3B4OwogICAgICAgICAgZm9udC1mYW1pbHk6IHRp bWVzIG5ldyByb21hbixuZXcgeW9yayx0aW1lcyxzZXJpZjsKICAgICAgICAgIGJhY2tncm91bmQt Y29sb3I6IHRyYW5zcGFyZW50OyBmb250LXN0eWxlOiBub3JtYWw7IiBjbGFzcz0iIj48YnI+CiAg ICAgICAgPC9kaXY+CiAgICAgICAgPGRpdiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9u dC1zaXplOiAxMC42NjY3cHg7CiAgICAgICAgICBmb250LWZhbWlseTogdGltZXMgbmV3IHJvbWFu LG5ldyB5b3JrLHRpbWVzLHNlcmlmOwogICAgICAgICAgYmFja2dyb3VuZC1jb2xvcjogdHJhbnNw YXJlbnQ7IGZvbnQtc3R5bGU6IG5vcm1hbDsiIGNsYXNzPSIiPkJ1dAogICAgICAgICAgSSBzdGls bCBoYXZlIHRoZSBiZWxvdyB3YXJuaW5ncyBpbiBteSBvdmlydC1lbmdpbmUubG9nOi08YnIKICAg ICAgICAgICAgY2xhc3M9IiIgc3R5bGU9IiI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRpdiBj bGFzcz0iIiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAxMC42NjY3cHg7 CiAgICAgICAgICBmb250LWZhbWlseTogdGltZXMgbmV3IHJvbWFuLG5ldyB5b3JrLHRpbWVzLHNl cmlmOwogICAgICAgICAgYmFja2dyb3VuZC1jb2xvcjogdHJhbnNwYXJlbnQ7IGZvbnQtc3R5bGU6 IG5vcm1hbDsiPjxicgogICAgICAgICAgICBzdHlsZT0iIiBjbGFzcz0iIj4KICAgICAgICA8L2Rp dj4KICAgICAgICA8ZGl2IGNsYXNzPSIiIHN0eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250 LXNpemU6IDEwLjY2NjdweDsKICAgICAgICAgIGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4s bmV3IHlvcmssdGltZXMsc2VyaWY7CiAgICAgICAgICBiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3Bh cmVudDsgZm9udC1zdHlsZTogbm9ybWFsOyI+MjAxNC0wOS0wMQogICAgICAgICAgMjM6MDk6NDcs NTUwIFdBUk4mbmJzcDsKICAgICAgICAgIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuYmxsLmdsdXN0 ZXIuR2x1c3RlclN5bmNKb2JdCiAgICAgICAgICAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3Jr ZXItMjYpIFZvbHVtZSBYWFggY29udGFpbnMgYQogICAgICAgICAgYXBwYXJlbnRseSBjb3JydXB0 IGJyaWNrKHMpLiBIZW5jZSB3aWxsIG5vdCBhZGQgaXQgdG8gZW5naW5lCiAgICAgICAgICBhdCB0 aGlzIHBvaW50LjxiciBzdHlsZT0iIiBjbGFzcz0iIj4KICAgICAgICAgIDIwMTQtMDktMDEgMjM6 MDk6NTIsNTU4IElORk8mbmJzcDsKICAgICAgICAgIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRz YnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0VkRTQ29tbWFuZF0KICAgICAgICAgIChE ZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci02MSkgU1RBUlQsCiAgICAgICAgICBHbHVzdGVy Vm9sdW1lc0xpc3RWRFNDb21tYW5kKEhvc3ROYW1lID0gaG9zdDEsIEhvc3RJZCA9CiAgICAgICAg ICAwZGUwZTdkNy1hM2RkLTQ3NzYtOTM5My1jMzZiOGM0YjQ1MmQpLCBsb2cgaWQ6IDJiNjM4MWIw PGJyCiAgICAgICAgICAgIHN0eWxlPSIiIGNsYXNzPSIiPgogICAgICAgICAgMjAxNC0wOS0wMSAy MzowOTo1Miw1ODggV0FSTiZuYnNwOwogICAgICAgICAgW29yZy5vdmlydC5lbmdpbmUuY29yZS52 ZHNicm9rZXIuZ2x1c3Rlci5HbHVzdGVyVm9sdW1lc0xpc3RSZXR1cm5Gb3JYbWxScGNdCiAgICAg ICAgICAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNjEpIENvdWxkIG5vdCBhZGQgYnJp Y2sKICAgICAgICAgIGhvc3QyOi92YXIvbGliL2V4cG9ydHMvZ2x1c3Rlci9icmljazEgdG8gdm9s dW1lCiAgICAgICAgICBmY2E5ZDhhNC0zOTJkLTRlNTUtYTJhYy04ZTA5YTc1YzVjZDIgLSBzZXJ2 ZXIgdXVpZAogICAgICAgICAgZDc3NjMyODMtMTBiNi00ODgwLWE3YjMtNDhkMDM1MzIwNWIwIG5v dCBmb3VuZCBpbiBjbHVzdGVyCiAgICAgICAgICAwMDAwMDAwMS0wMDAxLTAwMDEtMDAwMS0wMDAw MDAwMDAzM2E8YnIgc3R5bGU9IiIgY2xhc3M9IiI+CiAgICAgICAgICAyMDE0LTA5LTAxIDIzOjA5 OjUyLDU5NCBJTkZPJm5ic3A7CiAgICAgICAgICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jy b2tlci5nbHVzdGVyLkdsdXN0ZXJWb2x1bWVzTGlzdFZEU0NvbW1hbmRdCiAgICAgICAgICAoRGVm YXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNjEpIEZJTklTSCwKICAgICAgICAgIEdsdXN0ZXJW b2x1bWVzTGlzdFZEU0NvbW1hbmQsIHJldHVybjoKICAgICAgICAgIHtmY2E5ZDhhNC0zOTJkLTRl NTUtYTJhYy04ZTA5YTc1YzVjZDI9b3JnLm92aXJ0LmVuZ2luZS5jb3JlLmNvbW1vbi5idXNpbmVz c2VudGl0aWVzLmdsdXN0ZXIuR2x1c3RlclZvbHVtZUVudGl0eUAxMDcyNTRhY30sCiAgICAgICAg ICBsb2cgaWQ6IDJiNjM4MWIwPC9kaXY+CiAgICAgICAgPGRpdiBjbGFzcz0iIiBzdHlsZT0iY29s b3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAxMC42NjY3cHg7CiAgICAgICAgICBmb250LWZh bWlseTogdGltZXMgbmV3IHJvbWFuLG5ldyB5b3JrLHRpbWVzLHNlcmlmOwogICAgICAgICAgYmFj a2dyb3VuZC1jb2xvcjogdHJhbnNwYXJlbnQ7IGZvbnQtc3R5bGU6IG5vcm1hbDsiPjxicj4KICAg ICAgICA8L2Rpdj4KICAgICAgPC9kaXY+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+CiAgICA8 YnI+CiAgICBJdCBsb29rcyBsaWtlIHlvdSdyZSBtYW5hZ2luZyB5b3VyIGdsdXN0ZXIgc3RvcmFn ZSB1c2luZyBvVmlydCBhcwogICAgd2VsbC4gU2luY2Ugb25lIG9mIHlvdXIgaG9zdHMgZ290IGRp c2Nvbm5lY3RlZCAoaG9zdDIgPyksIGRpZCB0aGUKICAgIGhvc3QgZW50cnkgZ2V0IHJlbW92ZWQg ZnJvbSB0aGUgbGlzdCBvZiBob3N0cyBpbiB5b3VyIGNsdXN0ZXI/PGJyPgogICAgPGJyPgogICAg VGhlIGdsdXN0ZXIgdm9sdW1lIGhhcyBhIGJyaWNrIG9uIG9uZSBvZiB0aGUgcmVtb3ZlZCBob3N0 cywgYW5kIGl0CiAgICBpcyB1bmFibGUgdG8gc3luY2hyb25pemUgdGhlIGJyaWNrIGluZm9ybWF0 aW9uIHJldHVybmVkIGZyb20gZ2x1c3RlcgogICAgY2xpIHRvIHRoZSBlbmdpbmUncyBkYXRhYmFz ZS48YnI+CiAgICA8YnI+CiAgICA8YnI+CiAgICA8YnI+CiAgICA8YmxvY2txdW90ZQogICAgICBj aXRlPSJtaWQ6MTQwOTYwNDI1OC43MzExNi5ZYWhvb01haWxOZW9Ad2ViMTI0NTAzLm1haWwubmUx LnlhaG9vLmNvbSIKICAgICAgdHlwZT0iY2l0ZSI+CiAgICAgIDxkaXYgc3R5bGU9ImNvbG9yOiMw MDA7IGJhY2tncm91bmQtY29sb3I6I2ZmZjsgZm9udC1mYW1pbHk6dGltZXMKICAgICAgICBuZXcg cm9tYW4sIG5ldyB5b3JrLCB0aW1lcywgc2VyaWY7Zm9udC1zaXplOjhwdCI+CiAgICAgICAgPGRp diBjbGFzcz0iIiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAxMC42NjY3 cHg7CiAgICAgICAgICBmb250LWZhbWlseTogdGltZXMgbmV3IHJvbWFuLG5ldyB5b3JrLHRpbWVz LHNlcmlmOwogICAgICAgICAgYmFja2dyb3VuZC1jb2xvcjogdHJhbnNwYXJlbnQ7IGZvbnQtc3R5 bGU6IG5vcm1hbDsiPkJlc3QKICAgICAgICAgIFJlZ2FyZHM8L2Rpdj4KICAgICAgICA8ZGl2IGNs YXNzPSIiIHN0eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LXNpemU6IDEwLjY2NjdweDsK ICAgICAgICAgIGZvbnQtZmFtaWx5OiB0aW1lcyBuZXcgcm9tYW4sbmV3IHlvcmssdGltZXMsc2Vy aWY7CiAgICAgICAgICBiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVudDsgZm9udC1zdHlsZTog bm9ybWFsOyI+RmF3enkKICAgICAgICAgIElicmFoaW08YnI+CiAgICAgICAgPC9kaXY+CiAgICAg IDwvZGl2PgogICAgICA8YnI+CiAgICAgIDxmaWVsZHNldCBjbGFzcz0ibWltZUF0dGFjaG1lbnRI ZWFkZXIiPjwvZmllbGRzZXQ+CiAgICAgIDxicj4KICAgICAgPHByZSB3cmFwPSIiPl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClVzZXJzIG1haWxpbmcgbGlz dAo8YSBjbGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIiBocmVmPSJtYWlsdG86VXNlcnNA b3ZpcnQub3JnIj5Vc2Vyc0BvdmlydC5vcmc8L2E+CjxhIGNsYXNzPSJtb3otdHh0LWxpbmstZnJl ZXRleHQiIGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2Vy cyI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzPC9hPgo8L3By ZT4KICAgIDwvYmxvY2txdW90ZT4KICAgIDxicj4KICA8L2JvZHk+CjwvaHRtbD4KCi0tLS0tLS0t LS0tLS0tMDMwNzA3MDUwNDA5MDQwODA0MDAwNzA1LS0K --===============0659353303169605013==--