9:08 a.m.
Hey all,
since the org.ovirt.engine.core.common.job.Job/Step... has been moved to use the JPA we
have a problem on frontend.
The problem is that the @OneToMany annotations results in a List which is of type
PersistentBag. When we send this to
Frontend it fails during deserialization. It actually fails quite bad because the FE
already has an ui-override of it which is not correct resulting in
a ton of NPEs in development mode.
So, there are 2 nasty fixes I have made where none of them should be merged but
demonstrate the possibilities:
1: extend the FE to be able to work with the PersistentBag
(https://gerrit.ovirt.org/#/c/41682/)
not really good solution since the PersistenBag is an internal Hibernate class which is
really not meant to be passed around
2: fix on the backend to not send the PersistentBag but an ArrayList. This is only a PoC
fixed on a command we face the problem (https://gerrit.ovirt.org/#/c/41797/)
Obviously this is not going to work for other commands accessing the same Job nor for
other entities.
So, the first option is generic but very very bad. The second option should be used but
not sure how to do this in a cheep way (e.g. without using reflection to deep traverse
everything sent back to
frontend checking if it does not have a PersistentBag in it.
Any better ideas?
Thanx,
Tomas
9:16 a.m.
On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
Hey all,
since the org.ovirt.engine.core.common.job.Job/Step... has been moved to use
the JPA we have a problem on frontend. The problem is that the @OneToMany
annotations results in a List which is of type PersistentBag. When we send
this to Frontend it fails during deserialization. It actually fails quite
bad because the FE already has an ui-override of it which is not correct
resulting in a ton of NPEs in development mode.
So, there are 2 nasty fixes I have made where none of them should be merged
but demonstrate the possibilities: 1: extend the FE to be able to work with
the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not really good
solution since the PersistenBag is an internal Hibernate class which is
really not meant to be passed around
2: fix on the backend to not send the PersistentBag but an ArrayList. This
is only a PoC fixed on a command we face the problem
(https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going to work
for other commands accessing the same Job nor for other entities.
So, the first option is generic but very very bad. The second option should
be used but not sure how to do this in a cheep way (e.g. without using
reflection to deep traverse everything sent back to frontend checking if it
does not have a PersistentBag in it.
Tomas,
Thanks, I was investigating the same issue, I noticed it last Friday just
before leaving, so I was investigating the problem to see what was going on.
You are right we should not be sending PersistentBag to the frontend at all.
So how about we do a combination of [1] and [2], but instead of delegating in
[1] we actually simple throw an exception stating don't sent PersistentBag to
the front end. that way anyone inadvertently using it will be notified
immediately (since their code won't work).
Alexander
Any better ideas?
Thanx,
Tomas
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
9:33 a.m.
----- Original Message -----
From: "Alexander Wels" <awels(a)redhat.com>
To: devel(a)ovirt.org
Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran Zelkha"
<lzelkha(a)redhat.com>
Sent: Monday, June 1, 2015 3:16:34 PM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
> Hey all,
>
> since the org.ovirt.engine.core.common.job.Job/Step... has been moved to
> use
> the JPA we have a problem on frontend. The problem is that the @OneToMany
> annotations results in a List which is of type PersistentBag. When we send
> this to Frontend it fails during deserialization. It actually fails quite
> bad because the FE already has an ui-override of it which is not correct
> resulting in a ton of NPEs in development mode.
>
> So, there are 2 nasty fixes I have made where none of them should be merged
> but demonstrate the possibilities: 1: extend the FE to be able to work with
> the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not really good
> solution since the PersistenBag is an internal Hibernate class which is
> really not meant to be passed around
>
> 2: fix on the backend to not send the PersistentBag but an ArrayList. This
> is only a PoC fixed on a command we face the problem
> (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going to work
> for other commands accessing the same Job nor for other entities.
>
> So, the first option is generic but very very bad. The second option should
> be used but not sure how to do this in a cheep way (e.g. without using
> reflection to deep traverse everything sent back to frontend checking if it
> does not have a PersistentBag in it.
>
Tomas,
Thanks, I was investigating the same issue, I noticed it last Friday just
before leaving, so I was investigating the problem to see what was going on.
You are right we should not be sending PersistentBag to the frontend at all.
So how about we do a combination of [1] and [2], but instead of delegating in
[1] we actually simple throw an exception stating don't sent PersistentBag to
the front end. that way anyone inadvertently using it will be notified
immediately (since their code won't work).
Throwing the exception would help us in debugging but the main question is how will we
make it work?
Since we are planning to move more and more to JPA so we will face this issue more and
more often.
Solving it one by one on backend in each command is not going to work.
Alexander
> Any better ideas?
> Thanx,
> Tomas
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
9:56 a.m.
--Apple-Mail-EA5039A5-C9B4-4BAD-951C-B2D3B44C2B5B
Content-Type: text/plain;
charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Guys,
Why is https://gerrit.ovirt.org/#/c/40804/ not enough?=20
Where do you get these errors?
On Jun 1, 2015, at 4:33 PM, Tomas Jelinek <tjelinek(a)redhat.com>
wrote:
=20
=20
=20
----- Original Message -----
> From: "Alexander Wels" <awels(a)redhat.com>
> To: devel(a)ovirt.org
> Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran Zelkha"
<lzelkha@redhat=
.com>
> Sent: Monday, June 1, 2015 3:16:34 PM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>=20
>> On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
>> Hey all,
>>=20
>> since the org.ovirt.engine.core.common.job.Job/Step... has been moved to=
>> use
>> the JPA we have a problem on frontend. The problem is that the @OneToMan=
y
>> annotations results in a List which is of type PersistentBag.
When we se=
nd
>> this to Frontend it fails during deserialization. It actually
fails quit=
e
>> bad because the FE already has an ui-override of it which is
not correct=
>> resulting in a ton of NPEs in development mode.
>>=20
>> So, there are 2 nasty fixes I have made where none of them should be mer=
ged
>> but demonstrate the possibilities: 1: extend the FE to be
able to work w=
ith
>> the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not
really good
>> solution since the PersistenBag is an internal Hibernate class which is
>> really not meant to be passed around
>>=20
>> 2: fix on the backend to not send the PersistentBag but an ArrayList. Th=
is
>> is only a PoC fixed on a command we face the problem
>> (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going to wor=
k
>> for other commands accessing the same Job nor for other
entities.
>>=20
>> So, the first option is generic but very very bad. The second option sho=
uld
>> be used but not sure how to do this in a cheep way (e.g.
without using
>> reflection to deep traverse everything sent back to frontend checking if=
it
>> does not have a PersistentBag in it.
>=20
> Tomas,
>=20
> Thanks, I was investigating the same issue, I noticed it last Friday just=
> before leaving, so I was investigating the problem to see what
was going o=
n.
> You are right we should not be sending PersistentBag to the
frontend at a=
ll.
> So how about we do a combination of [1] and [2], but instead of
delegatin=
g in
> [1] we actually simple throw an exception stating don't sent
PersistentBa=
g to
> the front end. that way anyone inadvertently using it will be
notified
> immediately (since their code won't work).
=20
Throwing the exception would help us in debugging but the main question is=
how
will we make it work?
Since we are planning to move more and more to JPA so we will face
this is=
sue more and more often.=20
Solving it one by one on backend in each command is not going to
work.
=20
>=20
> Alexander
>=20
>> Any better ideas?
>> Thanx,
>> Tomas
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>=20
>=20
--Apple-Mail-EA5039A5-C9B4-4BAD-951C-B2D3B44C2B5B
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: base64
PGh0bWw+PGhlYWQ+PG1ldGEgaHR0cC1lcXVpdj0iY29udGVudC10eXBlIiBjb250ZW50PSJ0ZXh0
L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPjwvaGVhZD48Ym9keSBkaXI9ImF1dG8iPjxkaXY+R3V5cyw8
L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PldoeSBpcyZuYnNwOzxhIGhyZWY9Imh0dHBzOi8vZ2Vy
cml0Lm92aXJ0Lm9yZy8jL2MvNDA4MDQvIj5odHRwczovL2dlcnJpdC5vdmlydC5vcmcvIy9jLzQw
ODA0LzwvYT4mbmJzcDtub3QgZW5vdWdoPyZuYnNwOzwvZGl2PjxkaXY+V2hlcmUgZG8geW91IGdl
dCB0aGVzZSBlcnJvcnM/PGJyPjxicj48YnI+PC9kaXY+PGRpdj48YnI+T24gSnVuIDEsIDIwMTUs
IGF0IDQ6MzMgUE0sIFRvbWFzIEplbGluZWsgJmx0OzxhIGhyZWY9Im1haWx0bzp0amVsaW5la0By
ZWRoYXQuY29tIj50amVsaW5la0ByZWRoYXQuY29tPC9hPiZndDsgd3JvdGU6PGJyPjxicj48L2Rp
dj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48ZGl2PjxzcGFuPjwvc3Bhbj48YnI+PHNwYW4+PC9z
cGFuPjxicj48c3Bhbj4tLS0tLSBPcmlnaW5hbCBNZXNzYWdlIC0tLS0tPC9zcGFuPjxicj48Ymxv
Y2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj5Gcm9tOiAiQWxleGFuZGVyIFdlbHMiICZsdDs8YSBo
cmVmPSJtYWlsdG86YXdlbHNAcmVkaGF0LmNvbSI+YXdlbHNAcmVkaGF0LmNvbTwvYT4mZ3Q7PC9z
cGFuPjxicj48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+VG86IDxh
IGhyZWY9Im1haWx0bzpkZXZlbEBvdmlydC5vcmciPmRldmVsQG92aXJ0Lm9yZzwvYT48L3NwYW4+
PGJyPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj5DYzogIlRvbWFz
IEplbGluZWsiICZsdDs8YSBocmVmPSJtYWlsdG86dGplbGluZWtAcmVkaGF0LmNvbSI+dGplbGlu
ZWtAcmVkaGF0LmNvbTwvYT4mZ3Q7LCAiTGlyYW4gWmVsa2hhIiAmbHQ7PGEgaHJlZj0ibWFpbHRv
Omx6ZWxraGFAcmVkaGF0LmNvbSI+bHplbGtoYUByZWRoYXQuY29tPC9hPiZndDs8L3NwYW4+PGJy
PjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj5TZW50OiBNb25kYXks
IEp1bmUgMSwgMjAxNSAzOjE2OjM0IFBNPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PGJsb2NrcXVv
dGUgdHlwZT0iY2l0ZSI+PHNwYW4+U3ViamVjdDogUmU6IFtvdmlydC1kZXZlbF0gaGliZXJuYXRl
J3MgaW50ZXJuYWwgUGVyc2lzdGVudEJhZyBzZW50IHRvIEZFPC9zcGFuPjxicj48L2Jsb2NrcXVv
dGU+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+PC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+
PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+T24gTW9uZGF5LCBKdW5lIDAxLCAyMDE1IDA5
OjA4OjUwIEFNIFRvbWFzIEplbGluZWsgd3JvdGU6PC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PGJs
b2NrcXVvdGUgdHlwZT0iY2l0ZSI+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+SGV5IGFs
bCw8L3NwYW4+PGJyPjwvYmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUgdHlwZT0i
Y2l0ZSI+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+PC9zcGFuPjxicj48L2Jsb2NrcXVv
dGU+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxibG9ja3F1b3RlIHR5cGU9
ImNpdGUiPjxzcGFuPnNpbmNlIHRoZSBvcmcub3ZpcnQuZW5naW5lLmNvcmUuY29tbW9uLmpvYi5K
b2IvU3RlcC4uLiBoYXMgYmVlbiBtb3ZlZCB0bzwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjwvYmxv
Y2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48
c3Bhbj51c2U8L3NwYW4+PGJyPjwvYmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUg
dHlwZT0iY2l0ZSI+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+dGhlIEpQQSB3ZSBoYXZl
IGEgcHJvYmxlbSBvbiBmcm9udGVuZC4gVGhlIHByb2JsZW0gaXMgdGhhdCB0aGUgQE9uZVRvTWFu
eTwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJj
aXRlIj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj5hbm5vdGF0aW9ucyByZXN1bHRzIGlu
IGEgTGlzdCB3aGljaCBpcyBvZiB0eXBlIFBlcnNpc3RlbnRCYWcuIFdoZW4gd2Ugc2VuZDwvc3Bh
bj48YnI+PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48
YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj50aGlzIHRvIEZyb250ZW5kIGl0IGZhaWxzIGR1
cmluZyBkZXNlcmlhbGl6YXRpb24uIEl0IGFjdHVhbGx5IGZhaWxzIHF1aXRlPC9zcGFuPjxicj48
L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxibG9ja3F1
b3RlIHR5cGU9ImNpdGUiPjxzcGFuPmJhZCBiZWNhdXNlIHRoZSBGRSBhbHJlYWR5IGhhcyBhbiB1
aS1vdmVycmlkZSBvZiBpdCB3aGljaCBpcyBub3QgY29ycmVjdDwvc3Bhbj48YnI+PC9ibG9ja3F1
b3RlPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48YmxvY2txdW90ZSB0eXBl
PSJjaXRlIj48c3Bhbj5yZXN1bHRpbmcgaW4gYSB0b24gb2YgTlBFcyBpbiBkZXZlbG9wbWVudCBt
b2RlLjwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBl
PSJjaXRlIj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj48L3NwYW4+PGJyPjwvYmxvY2tx
dW90ZT48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PGJsb2NrcXVvdGUgdHlw
ZT0iY2l0ZSI+PHNwYW4+U28sIHRoZXJlIGFyZSAyIG5hc3R5IGZpeGVzIEkgaGF2ZSBtYWRlIHdo
ZXJlIG5vbmUgb2YgdGhlbSBzaG91bGQgYmUgbWVyZ2VkPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+
PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxibG9ja3F1b3RlIHR5cGU9ImNp
dGUiPjxzcGFuPmJ1dCBkZW1vbnN0cmF0ZSB0aGUgcG9zc2liaWxpdGllczogMTogZXh0ZW5kIHRo
ZSBGRSB0byBiZSBhYmxlIHRvIHdvcmsgd2l0aDwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjwvYmxv
Y2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48
c3Bhbj50aGUgUGVyc2lzdGVudEJhZyAoPGEgaHJlZj0iaHR0cHM6Ly9nZXJyaXQub3ZpcnQub3Jn
LyMvYy80MTY4Mi8iPmh0dHBzOi8vZ2Vycml0Lm92aXJ0Lm9yZy8jL2MvNDE2ODIvPC9hPikgbm90
IHJlYWxseSBnb29kPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3RlPjxibG9ja3F1
b3RlIHR5cGU9ImNpdGUiPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPnNvbHV0aW9uIHNp
bmNlIHRoZSBQZXJzaXN0ZW5CYWcgaXMgYW4gaW50ZXJuYWwgSGliZXJuYXRlIGNsYXNzIHdoaWNo
IGlzPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9
ImNpdGUiPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPnJlYWxseSBub3QgbWVhbnQgdG8g
YmUgcGFzc2VkIGFyb3VuZDwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48Ymxv
Y2txdW90ZSB0eXBlPSJjaXRlIj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj48L3NwYW4+
PGJyPjwvYmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PGJs
b2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+MjogZml4IG9uIHRoZSBiYWNrZW5kIHRvIG5vdCBz
ZW5kIHRoZSBQZXJzaXN0ZW50QmFnIGJ1dCBhbiBBcnJheUxpc3QuIFRoaXM8L3NwYW4+PGJyPjwv
YmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PGJsb2NrcXVv
dGUgdHlwZT0iY2l0ZSI+PHNwYW4+aXMgb25seSBhIFBvQyBmaXhlZCBvbiBhIGNvbW1hbmQgd2Ug
ZmFjZSB0aGUgcHJvYmxlbTwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48Ymxv
Y2txdW90ZSB0eXBlPSJjaXRlIj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj4oPGEgaHJl
Zj0iaHR0cHM6Ly9nZXJyaXQub3ZpcnQub3JnLyMvYy80MTc5Ny8iPmh0dHBzOi8vZ2Vycml0Lm92
aXJ0Lm9yZy8jL2MvNDE3OTcvPC9hPikgT2J2aW91c2x5IHRoaXMgaXMgbm90IGdvaW5nIHRvIHdv
cms8L3NwYW4+PGJyPjwvYmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUgdHlwZT0i
Y2l0ZSI+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+Zm9yIG90aGVyIGNvbW1hbmRzIGFj
Y2Vzc2luZyB0aGUgc2FtZSBKb2Igbm9yIGZvciBvdGhlciBlbnRpdGllcy48L3NwYW4+PGJyPjwv
YmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PGJsb2NrcXVv
dGUgdHlwZT0iY2l0ZSI+PHNwYW4+PC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3Rl
PjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPlNv
LCB0aGUgZmlyc3Qgb3B0aW9uIGlzIGdlbmVyaWMgYnV0IHZlcnkgdmVyeSBiYWQuIFRoZSBzZWNv
bmQgb3B0aW9uIHNob3VsZDwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48Ymxv
Y2txdW90ZSB0eXBlPSJjaXRlIj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj5iZSB1c2Vk
IGJ1dCBub3Qgc3VyZSBob3cgdG8gZG8gdGhpcyBpbiBhIGNoZWVwIHdheSAoZS5nLiB3aXRob3V0
IHVzaW5nPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5
cGU9ImNpdGUiPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPnJlZmxlY3Rpb24gdG8gZGVl
cCB0cmF2ZXJzZSBldmVyeXRoaW5nIHNlbnQgYmFjayB0byBmcm9udGVuZCBjaGVja2luZyBpZiBp
dDwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJj
aXRlIj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj5kb2VzIG5vdCBoYXZlIGEgUGVyc2lz
dGVudEJhZyBpbiBpdC48L3NwYW4+PGJyPjwvYmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+PGJsb2Nr
cXVvdGUgdHlwZT0iY2l0ZSI+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+PC9zcGFuPjxi
cj48L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFu
Pjwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPlRv
bWFzLDwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFu
Pjwvc3Bhbj48YnI+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPlRo
YW5rcywgSSB3YXMgaW52ZXN0aWdhdGluZyB0aGUgc2FtZSBpc3N1ZSwgSSBub3RpY2VkIGl0IGxh
c3QgRnJpZGF5IGp1c3Q8L3NwYW4+PGJyPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJj
aXRlIj48c3Bhbj5iZWZvcmUgbGVhdmluZywgc28gSSB3YXMgaW52ZXN0aWdhdGluZyB0aGUgcHJv
YmxlbSB0byBzZWUgd2hhdCB3YXMgZ29pbmcgb24uPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PGJs
b2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+WW91IGFyZSByaWdodCB3ZSBzaG91bGQgbm90IGJl
IHNlbmRpbmcgUGVyc2lzdGVudEJhZyB0byB0aGUgZnJvbnRlbmQgYXQgYWxsLjwvc3Bhbj48YnI+
PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPlNvIGhvdyBhYm91dCB3
ZSBkbyBhIGNvbWJpbmF0aW9uIG9mIFsxXSBhbmQgWzJdLCBidXQgaW5zdGVhZCBvZiBkZWxlZ2F0
aW5nIGluPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNw
YW4+WzFdIHdlIGFjdHVhbGx5IHNpbXBsZSB0aHJvdyBhbiBleGNlcHRpb24gc3RhdGluZyBkb24n
dCBzZW50IFBlcnNpc3RlbnRCYWcgdG88L3NwYW4+PGJyPjwvYmxvY2txdW90ZT48YmxvY2txdW90
ZSB0eXBlPSJjaXRlIj48c3Bhbj50aGUgZnJvbnQgZW5kLiB0aGF0IHdheSBhbnlvbmUgaW5hZHZl
cnRlbnRseSB1c2luZyBpdCB3aWxsIGJlIG5vdGlmaWVkPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+
PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+aW1tZWRpYXRlbHkgKHNpbmNlIHRoZWlyIGNv
ZGUgd29uJ3Qgd29yaykuPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PHNwYW4+PC9zcGFuPjxicj48
c3Bhbj5UaHJvd2luZyB0aGUgZXhjZXB0aW9uIHdvdWxkIGhlbHAgdXMgaW4gZGVidWdnaW5nIGJ1
dCB0aGUgbWFpbiBxdWVzdGlvbiBpcyBob3cgd2lsbCB3ZSBtYWtlIGl0IHdvcms/PC9zcGFuPjxi
cj48c3Bhbj5TaW5jZSB3ZSBhcmUgcGxhbm5pbmcgdG8gbW92ZSBtb3JlIGFuZCBtb3JlIHRvIEpQ
QSBzbyB3ZSB3aWxsIGZhY2UgdGhpcyBpc3N1ZSBtb3JlIGFuZCBtb3JlIG9mdGVuLiA8L3NwYW4+
PGJyPjxzcGFuPlNvbHZpbmcgaXQgb25lIGJ5IG9uZSBvbiBiYWNrZW5kIGluIGVhY2ggY29tbWFu
ZCBpcyBub3QgZ29pbmcgdG8gd29yay48L3NwYW4+PGJyPjxzcGFuPjwvc3Bhbj48YnI+PGJsb2Nr
cXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+PC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PGJsb2NrcXVv
dGUgdHlwZT0iY2l0ZSI+PHNwYW4+QWxleGFuZGVyPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PGJs
b2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+PC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PGJsb2Nr
cXVvdGUgdHlwZT0iY2l0ZSI+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+QW55IGJldHRl
ciBpZGVhcz88L3NwYW4+PGJyPjwvYmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUg
dHlwZT0iY2l0ZSI+PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+VGhhbngsPC9zcGFuPjxi
cj48L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxibG9j
a3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPlRvbWFzPC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PC9i
bG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUi
PjxzcGFuPl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPC9z
cGFuPjxicj48L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3RlPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUi
PjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPkRldmVsIG1haWxpbmcgbGlzdDwvc3Bhbj48
YnI+PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48Ymxv
Y2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj48YSBocmVmPSJtYWlsdG86RGV2ZWxAb3ZpcnQub3Jn
Ij5EZXZlbEBvdmlydC5vcmc8L2E+PC9zcGFuPjxicj48L2Jsb2NrcXVvdGU+PC9ibG9ja3F1b3Rl
PjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPjxzcGFuPjxh
IGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby9kZXZlbCI+aHR0
cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RldmVsPC9hPjwvc3Bhbj48YnI+
PC9ibG9ja3F1b3RlPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj48
L3NwYW4+PGJyPjwvYmxvY2txdW90ZT48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj48L3Nw
YW4+PGJyPjwvYmxvY2txdW90ZT48L2Rpdj48L2Jsb2NrcXVvdGU+PC9ib2R5PjwvaHRtbD4=
--Apple-Mail-EA5039A5-C9B4-4BAD-951C-B2D3B44C2B5B--
9:59 a.m.
----- Original Message -----
From: "Liran Zelkha" <lzelkha(a)redhat.com>
To: "Tomas Jelinek" <tjelinek(a)redhat.com>
Cc: awels(a)redhat.com, devel(a)ovirt.org
Sent: Monday, June 1, 2015 3:56:03 PM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
Guys,
Why is https://gerrit.ovirt.org/#/c/40804/ not enough?
because the PersistenBag has an inner property "bag" and FE fails on bunch of
NPEs because of it
Where do you get these errors?
open debug mode and load webadmin - you will see loads of NPEs
> On Jun 1, 2015, at 4:33 PM, Tomas Jelinek <tjelinek(a)redhat.com> wrote:
>
>
>
> ----- Original Message -----
>> From: "Alexander Wels" <awels(a)redhat.com>
>> To: devel(a)ovirt.org
>> Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran
Zelkha"
>> <lzelkha(a)redhat.com>
>> Sent: Monday, June 1, 2015 3:16:34 PM
>> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>>
>>> On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
>>> Hey all,
>>>
>>> since the org.ovirt.engine.core.common.job.Job/Step... has been moved to
>>> use
>>> the JPA we have a problem on frontend. The problem is that the @OneToMany
>>> annotations results in a List which is of type PersistentBag. When we
>>> send
>>> this to Frontend it fails during deserialization. It actually fails quite
>>> bad because the FE already has an ui-override of it which is not correct
>>> resulting in a ton of NPEs in development mode.
>>>
>>> So, there are 2 nasty fixes I have made where none of them should be
>>> merged
>>> but demonstrate the possibilities: 1: extend the FE to be able to work
>>> with
>>> the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not really good
>>> solution since the PersistenBag is an internal Hibernate class which is
>>> really not meant to be passed around
>>>
>>> 2: fix on the backend to not send the PersistentBag but an ArrayList.
>>> This
>>> is only a PoC fixed on a command we face the problem
>>> (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going to work
>>> for other commands accessing the same Job nor for other entities.
>>>
>>> So, the first option is generic but very very bad. The second option
>>> should
>>> be used but not sure how to do this in a cheep way (e.g. without using
>>> reflection to deep traverse everything sent back to frontend checking if
>>> it
>>> does not have a PersistentBag in it.
>>
>> Tomas,
>>
>> Thanks, I was investigating the same issue, I noticed it last Friday just
>> before leaving, so I was investigating the problem to see what was going
>> on.
>> You are right we should not be sending PersistentBag to the frontend at
>> all.
>> So how about we do a combination of [1] and [2], but instead of delegating
>> in
>> [1] we actually simple throw an exception stating don't sent PersistentBag
>> to
>> the front end. that way anyone inadvertently using it will be notified
>> immediately (since their code won't work).
>
> Throwing the exception would help us in debugging but the main question is
> how will we make it work?
> Since we are planning to move more and more to JPA so we will face this
> issue more and more often.
> Solving it one by one on backend in each command is not going to work.
>
>>
>> Alexander
>>
>>> Any better ideas?
>>> Thanx,
>>> Tomas
>>> _______________________________________________
>>> Devel mailing list
>>> Devel(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>>
10:09 a.m.
On Jun 1, 2015, at 15:59 , Tomas Jelinek <tjelinek(a)redhat.com> wrote:
----- Original Message -----
> From: "Liran Zelkha" <lzelkha(a)redhat.com>
> To: "Tomas Jelinek" <tjelinek(a)redhat.com>
> Cc: awels(a)redhat.com, devel(a)ovirt.org
> Sent: Monday, June 1, 2015 3:56:03 PM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
> Guys,
>
> Why is https://gerrit.ovirt.org/#/c/40804/ not enough?
because the PersistenBag has an inner property "bag" and FE fails on bunch of
NPEs because of it
> Where do you get these errors?
open debug mode and load webadmin - you will see loads of NPEs
does this happen in normal mode too?
>
>
>
>> On Jun 1, 2015, at 4:33 PM, Tomas Jelinek <tjelinek(a)redhat.com> wrote:
>>
>>
>>
>> ----- Original Message -----
>>> From: "Alexander Wels" <awels(a)redhat.com>
>>> To: devel(a)ovirt.org
>>> Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran
Zelkha"
>>> <lzelkha(a)redhat.com>
>>> Sent: Monday, June 1, 2015 3:16:34 PM
>>> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>>>
>>>> On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
>>>> Hey all,
>>>>
>>>> since the org.ovirt.engine.core.common.job.Job/Step... has been moved to
>>>> use
>>>> the JPA we have a problem on frontend. The problem is that the
@OneToMany
>>>> annotations results in a List which is of type PersistentBag. When we
>>>> send
>>>> this to Frontend it fails during deserialization. It actually fails
quite
>>>> bad because the FE already has an ui-override of it which is not correct
>>>> resulting in a ton of NPEs in development mode.
>>>>
>>>> So, there are 2 nasty fixes I have made where none of them should be
>>>> merged
>>>> but demonstrate the possibilities: 1: extend the FE to be able to work
>>>> with
>>>> the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not really good
>>>> solution since the PersistenBag is an internal Hibernate class which is
>>>> really not meant to be passed around
>>>>
>>>> 2: fix on the backend to not send the PersistentBag but an ArrayList.
>>>> This
>>>> is only a PoC fixed on a command we face the problem
>>>> (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going to
work
>>>> for other commands accessing the same Job nor for other entities.
>>>>
>>>> So, the first option is generic but very very bad. The second option
>>>> should
>>>> be used but not sure how to do this in a cheep way (e.g. without using
>>>> reflection to deep traverse everything sent back to frontend checking if
>>>> it
>>>> does not have a PersistentBag in it.
>>>
>>> Tomas,
>>>
>>> Thanks, I was investigating the same issue, I noticed it last Friday just
>>> before leaving, so I was investigating the problem to see what was going
>>> on.
>>> You are right we should not be sending PersistentBag to the frontend at
>>> all.
>>> So how about we do a combination of [1] and [2], but instead of delegating
>>> in
>>> [1] we actually simple throw an exception stating don't sent
PersistentBag
>>> to
>>> the front end. that way anyone inadvertently using it will be notified
>>> immediately (since their code won't work).
>>
>> Throwing the exception would help us in debugging but the main question is
>> how will we make it work?
>> Since we are planning to move more and more to JPA so we will face this
>> issue more and more often.
>> Solving it one by one on backend in each command is not going to work.
>>
>>>
>>> Alexander
>>>
>>>> Any better ideas?
>>>> Thanx,
>>>> Tomas
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>>
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
10:16 a.m.
On Monday, June 01, 2015 04:09:08 PM Michal Skrivanek wrote:
On Jun 1, 2015, at 15:59 , Tomas Jelinek <tjelinek(a)redhat.com>
wrote:
> ----- Original Message -----
>
>> From: "Liran Zelkha" <lzelkha(a)redhat.com>
>> To: "Tomas Jelinek" <tjelinek(a)redhat.com>
>> Cc: awels(a)redhat.com, devel(a)ovirt.org
>> Sent: Monday, June 1, 2015 3:56:03 PM
>> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>>
>> Guys,
>>
>> Why is https://gerrit.ovirt.org/#/c/40804/ not enough?
>
> because the PersistenBag has an inner property "bag" and FE fails on
bunch
> of NPEs because of it>
>> Where do you get these errors?
>
> open debug mode and load webadmin - you will see loads of NPEs
does this happen in normal mode too?
It does not appear to happen in Chrome, but I have some open bugzillas about
unexplained null pointer exceptions in IE (the null pointers are during
transfer of objects), which may or may not be related to this problem.
Regardless doing any work in the GWT debugger is impossible right now due to
these exceptions.
>>> On Jun 1, 2015, at 4:33 PM, Tomas Jelinek
<tjelinek(a)redhat.com> wrote:
>>>
>>>
>>>
>>> ----- Original Message -----
>>>
>>>> From: "Alexander Wels" <awels(a)redhat.com>
>>>> To: devel(a)ovirt.org
>>>> Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran
Zelkha"
>>>> <lzelkha(a)redhat.com>
>>>> Sent: Monday, June 1, 2015 3:16:34 PM
>>>> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent
to
>>>> FE
>>>>
>>>>> On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
>>>>> Hey all,
>>>>>
>>>>> since the org.ovirt.engine.core.common.job.Job/Step... has been
moved
>>>>> to
>>>>> use
>>>>> the JPA we have a problem on frontend. The problem is that the
>>>>> @OneToMany
>>>>> annotations results in a List which is of type PersistentBag. When
we
>>>>> send
>>>>> this to Frontend it fails during deserialization. It actually fails
>>>>> quite
>>>>> bad because the FE already has an ui-override of it which is not
>>>>> correct
>>>>> resulting in a ton of NPEs in development mode.
>>>>>
>>>>> So, there are 2 nasty fixes I have made where none of them should
be
>>>>> merged
>>>>> but demonstrate the possibilities: 1: extend the FE to be able to
work
>>>>> with
>>>>> the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not really
>>>>> good
>>>>> solution since the PersistenBag is an internal Hibernate class
which
>>>>> is
>>>>> really not meant to be passed around
>>>>>
>>>>> 2: fix on the backend to not send the PersistentBag but an
ArrayList.
>>>>> This
>>>>> is only a PoC fixed on a command we face the problem
>>>>> (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going
to
>>>>> work
>>>>> for other commands accessing the same Job nor for other entities.
>>>>>
>>>>> So, the first option is generic but very very bad. The second
option
>>>>> should
>>>>> be used but not sure how to do this in a cheep way (e.g. without
using
>>>>> reflection to deep traverse everything sent back to frontend
checking
>>>>> if
>>>>> it
>>>>> does not have a PersistentBag in it.
>>>>
>>>> Tomas,
>>>>
>>>> Thanks, I was investigating the same issue, I noticed it last Friday
>>>> just
>>>> before leaving, so I was investigating the problem to see what was
>>>> going
>>>> on.
>>>> You are right we should not be sending PersistentBag to the frontend at
>>>> all.
>>>> So how about we do a combination of [1] and [2], but instead of
>>>> delegating
>>>> in
>>>> [1] we actually simple throw an exception stating don't sent
>>>> PersistentBag
>>>> to
>>>> the front end. that way anyone inadvertently using it will be notified
>>>> immediately (since their code won't work).
>>>
>>> Throwing the exception would help us in debugging but the main question
>>> is
>>> how will we make it work?
>>> Since we are planning to move more and more to JPA so we will face this
>>> issue more and more often.
>>> Solving it one by one on backend in each command is not going to work.
>>>
>>>> Alexander
>>>>
>>>>> Any better ideas?
>>>>> Thanx,
>>>>> Tomas
>>>>> _______________________________________________
>>>>> Devel mailing list
>>>>> Devel(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
10:19 a.m.
On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
----- Original Message -----
> From: "Alexander Wels" <awels(a)redhat.com>
> To: devel(a)ovirt.org
> Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran Zelkha"
> <lzelkha(a)redhat.com> Sent: Monday, June 1, 2015 3:16:34 PM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
> On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
> > Hey all,
> >
> > since the org.ovirt.engine.core.common.job.Job/Step... has been moved to
> > use
> > the JPA we have a problem on frontend. The problem is that the
> > @OneToMany
> > annotations results in a List which is of type PersistentBag. When we
> > send
> > this to Frontend it fails during deserialization. It actually fails
> > quite
> > bad because the FE already has an ui-override of it which is not correct
> > resulting in a ton of NPEs in development mode.
> >
> > So, there are 2 nasty fixes I have made where none of them should be
> > merged
> > but demonstrate the possibilities: 1: extend the FE to be able to work
> > with
> > the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not really good
> > solution since the PersistenBag is an internal Hibernate class which is
> > really not meant to be passed around
> >
> > 2: fix on the backend to not send the PersistentBag but an ArrayList.
> > This
> > is only a PoC fixed on a command we face the problem
> > (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going to
> > work
> > for other commands accessing the same Job nor for other entities.
> >
> > So, the first option is generic but very very bad. The second option
> > should
> > be used but not sure how to do this in a cheep way (e.g. without using
> > reflection to deep traverse everything sent back to frontend checking if
> > it
> > does not have a PersistentBag in it.
>
> Tomas,
>
> Thanks, I was investigating the same issue, I noticed it last Friday just
> before leaving, so I was investigating the problem to see what was going
> on. You are right we should not be sending PersistentBag to the frontend
> at all. So how about we do a combination of [1] and [2], but instead of
> delegating in [1] we actually simple throw an exception stating don't
> sent PersistentBag to the front end. that way anyone inadvertently using
> it will be notified immediately (since their code won't work).
Throwing the exception would help us in debugging but the main question is
how will we make it work? Since we are planning to move more and more to
JPA so we will face this issue more and more often. Solving it one by one
on backend in each command is not going to work.
How about something like this [1]? It appears to use some aspects to translate
the hibernate internal classes into normal java util classes.
[1] https://code.google.com/p/dehibernator/
> Alexander
>
> > Any better ideas?
> > Thanx,
> > Tomas
> > _______________________________________________
> > Devel mailing list
> > Devel(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
10:51 a.m.
----- Original Message -----
From: "Alexander Wels" <awels(a)redhat.com>
To: "Tomas Jelinek" <tjelinek(a)redhat.com>
Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
Sent: Monday, June 1, 2015 4:19:47 PM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
> ----- Original Message -----
>
> > From: "Alexander Wels" <awels(a)redhat.com>
> > To: devel(a)ovirt.org
> > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran
Zelkha"
> > <lzelkha(a)redhat.com> Sent: Monday, June 1, 2015 3:16:34 PM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> > On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
> > > Hey all,
> > >
> > > since the org.ovirt.engine.core.common.job.Job/Step... has been moved
> > > to
> > > use
> > > the JPA we have a problem on frontend. The problem is that the
> > > @OneToMany
> > > annotations results in a List which is of type PersistentBag. When we
> > > send
> > > this to Frontend it fails during deserialization. It actually fails
> > > quite
> > > bad because the FE already has an ui-override of it which is not
> > > correct
> > > resulting in a ton of NPEs in development mode.
> > >
> > > So, there are 2 nasty fixes I have made where none of them should be
> > > merged
> > > but demonstrate the possibilities: 1: extend the FE to be able to work
> > > with
> > > the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not really good
> > > solution since the PersistenBag is an internal Hibernate class which is
> > > really not meant to be passed around
> > >
> > > 2: fix on the backend to not send the PersistentBag but an ArrayList.
> > > This
> > > is only a PoC fixed on a command we face the problem
> > > (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going to
> > > work
> > > for other commands accessing the same Job nor for other entities.
> > >
> > > So, the first option is generic but very very bad. The second option
> > > should
> > > be used but not sure how to do this in a cheep way (e.g. without using
> > > reflection to deep traverse everything sent back to frontend checking
> > > if
> > > it
> > > does not have a PersistentBag in it.
> >
> > Tomas,
> >
> > Thanks, I was investigating the same issue, I noticed it last Friday just
> > before leaving, so I was investigating the problem to see what was going
> > on. You are right we should not be sending PersistentBag to the frontend
> > at all. So how about we do a combination of [1] and [2], but instead of
> > delegating in [1] we actually simple throw an exception stating don't
> > sent PersistentBag to the front end. that way anyone inadvertently using
> > it will be notified immediately (since their code won't work).
>
> Throwing the exception would help us in debugging but the main question is
> how will we make it work? Since we are planning to move more and more to
> JPA so we will face this issue more and more often. Solving it one by one
> on backend in each command is not going to work.
How about something like this [1]? It appears to use some aspects to
translate
the hibernate internal classes into normal java util classes.
[1] https://code.google.com/p/dehibernator/
Well, not sure about that from performance point of view.
If we really move away from GWT completely in 4.0, then implementing all
hibernate inner collections in "uioverride" in a similar way as Tomas did
seems to me as the best solution from performance and "easiest to remove
when not needed" point of view.
> > Alexander
> >
> > > Any better ideas?
> > > Thanx,
> > > Tomas
> > > _______________________________________________
> > > Devel mailing list
> > > Devel(a)ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
9:16 p.m.
On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
----- Original Message -----
> From: "Alexander Wels" <awels(a)redhat.com>
> To: "Tomas Jelinek" <tjelinek(a)redhat.com>
> Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> Sent: Monday, June 1, 2015 4:19:47 PM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
> On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
> > ----- Original Message -----
> >
> > > From: "Alexander Wels" <awels(a)redhat.com>
> > > To: devel(a)ovirt.org
> > > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran
Zelkha"
> > > <lzelkha(a)redhat.com> Sent: Monday, June 1, 2015 3:16:34 PM
> > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to
> > > FE
> > >
> > > On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
> > > > Hey all,
> > > >
> > > > since the org.ovirt.engine.core.common.job.Job/Step... has been
> > > > moved
> > > > to
> > > > use
> > > > the JPA we have a problem on frontend. The problem is that the
> > > > @OneToMany
> > > > annotations results in a List which is of type PersistentBag. When
> > > > we
> > > > send
> > > > this to Frontend it fails during deserialization. It actually fails
> > > > quite
> > > > bad because the FE already has an ui-override of it which is not
> > > > correct
> > > > resulting in a ton of NPEs in development mode.
> > > >
> > > > So, there are 2 nasty fixes I have made where none of them should be
> > > > merged
> > > > but demonstrate the possibilities: 1: extend the FE to be able to
> > > > work
> > > > with
> > > > the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not really
> > > > good
> > > > solution since the PersistenBag is an internal Hibernate class which
> > > > is
> > > > really not meant to be passed around
> > > >
> > > > 2: fix on the backend to not send the PersistentBag but an
> > > > ArrayList.
> > > > This
> > > > is only a PoC fixed on a command we face the problem
> > > > (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not going to
> > > > work
> > > > for other commands accessing the same Job nor for other entities.
> > > >
> > > > So, the first option is generic but very very bad. The second option
> > > > should
> > > > be used but not sure how to do this in a cheep way (e.g. without
> > > > using
> > > > reflection to deep traverse everything sent back to frontend
> > > > checking
> > > > if
> > > > it
> > > > does not have a PersistentBag in it.
> > >
> > > Tomas,
> > >
> > > Thanks, I was investigating the same issue, I noticed it last Friday
> > > just
> > > before leaving, so I was investigating the problem to see what was
> > > going
> > > on. You are right we should not be sending PersistentBag to the
> > > frontend
> > > at all. So how about we do a combination of [1] and [2], but instead
> > > of
> > > delegating in [1] we actually simple throw an exception stating don't
> > > sent PersistentBag to the front end. that way anyone inadvertently
> > > using
> > > it will be notified immediately (since their code won't work).
> >
> > Throwing the exception would help us in debugging but the main question
> > is
> > how will we make it work? Since we are planning to move more and more to
> > JPA so we will face this issue more and more often. Solving it one by
> > one
> > on backend in each command is not going to work.
>
> How about something like this [1]? It appears to use some aspects to
> translate
> the hibernate internal classes into normal java util classes.
>
> [1] https://code.google.com/p/dehibernator/
Well, not sure about that from performance point of view.
If we really move away from GWT completely in 4.0, then implementing all
hibernate inner collections in "uioverride" in a similar way as Tomas did
seems to me as the best solution from performance and "easiest to remove
when not needed" point of view.
I took some inspiration from that code and wrote my own fairly simple
hibernate persistent collection replacer, and I put the patch up here [3]. It
still uses reflection so it is probably not the fastest thing ever written.
[3] https://gerrit.ovirt.org/#/c/41810/
> > > Alexander
> > >
> > > > Any better ideas?
> > > > Thanx,
> > > > Tomas
> > > > _______________________________________________
> > > > Devel mailing list
> > > > Devel(a)ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/devel
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
2:35 a.m.
top-posting:
Due to the fact that we'll see that all over the place, I really think that it would
be best to support that at the frontend level, and not the backend level.
Doing it in the backend level will cause a lot of overhead.
I'll leave the technical details to the UX experts here to see what's the right
approach to do it in the frontend side.
Thanks,
Oved
----- Original Message -----
From: "Alexander Wels" <awels(a)redhat.com>
To: "Martin Perina" <mperina(a)redhat.com>
Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
Sent: Tuesday, June 2, 2015 4:16:31 AM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> ----- Original Message -----
>
> > From: "Alexander Wels" <awels(a)redhat.com>
> > To: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> > Sent: Monday, June 1, 2015 4:19:47 PM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> > On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
> > > ----- Original Message -----
> > >
> > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > To: devel(a)ovirt.org
> > > > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>,
"Liran Zelkha"
> > > > <lzelkha(a)redhat.com> Sent: Monday, June 1, 2015 3:16:34 PM
> > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag
sent to
> > > > FE
> > > >
> > > > On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
> > > > > Hey all,
> > > > >
> > > > > since the org.ovirt.engine.core.common.job.Job/Step... has been
> > > > > moved
> > > > > to
> > > > > use
> > > > > the JPA we have a problem on frontend. The problem is that the
> > > > > @OneToMany
> > > > > annotations results in a List which is of type PersistentBag.
When
> > > > > we
> > > > > send
> > > > > this to Frontend it fails during deserialization. It actually
fails
> > > > > quite
> > > > > bad because the FE already has an ui-override of it which is
not
> > > > > correct
> > > > > resulting in a ton of NPEs in development mode.
> > > > >
> > > > > So, there are 2 nasty fixes I have made where none of them
should
> > > > > be
> > > > > merged
> > > > > but demonstrate the possibilities: 1: extend the FE to be able
to
> > > > > work
> > > > > with
> > > > > the PersistentBag (https://gerrit.ovirt.org/#/c/41682/) not
really
> > > > > good
> > > > > solution since the PersistenBag is an internal Hibernate class
> > > > > which
> > > > > is
> > > > > really not meant to be passed around
> > > > >
> > > > > 2: fix on the backend to not send the PersistentBag but an
> > > > > ArrayList.
> > > > > This
> > > > > is only a PoC fixed on a command we face the problem
> > > > > (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not
going
> > > > > to
> > > > > work
> > > > > for other commands accessing the same Job nor for other
entities.
> > > > >
> > > > > So, the first option is generic but very very bad. The second
> > > > > option
> > > > > should
> > > > > be used but not sure how to do this in a cheep way (e.g.
without
> > > > > using
> > > > > reflection to deep traverse everything sent back to frontend
> > > > > checking
> > > > > if
> > > > > it
> > > > > does not have a PersistentBag in it.
> > > >
> > > > Tomas,
> > > >
> > > > Thanks, I was investigating the same issue, I noticed it last Friday
> > > > just
> > > > before leaving, so I was investigating the problem to see what was
> > > > going
> > > > on. You are right we should not be sending PersistentBag to the
> > > > frontend
> > > > at all. So how about we do a combination of [1] and [2], but instead
> > > > of
> > > > delegating in [1] we actually simple throw an exception stating
don't
> > > > sent PersistentBag to the front end. that way anyone inadvertently
> > > > using
> > > > it will be notified immediately (since their code won't work).
> > >
> > > Throwing the exception would help us in debugging but the main question
> > > is
> > > how will we make it work? Since we are planning to move more and more
> > > to
> > > JPA so we will face this issue more and more often. Solving it one by
> > > one
> > > on backend in each command is not going to work.
> >
> > How about something like this [1]? It appears to use some aspects to
> > translate
> > the hibernate internal classes into normal java util classes.
> >
> > [1] https://code.google.com/p/dehibernator/
>
> Well, not sure about that from performance point of view.
>
> If we really move away from GWT completely in 4.0, then implementing all
> hibernate inner collections in "uioverride" in a similar way as Tomas did
> seems to me as the best solution from performance and "easiest to remove
> when not needed" point of view.
>
I took some inspiration from that code and wrote my own fairly simple
hibernate persistent collection replacer, and I put the patch up here [3]. It
still uses reflection so it is probably not the fastest thing ever written.
[3] https://gerrit.ovirt.org/#/c/41810/
> > > > Alexander
> > > >
> > > > > Any better ideas?
> > > > > Thanx,
> > > > > Tomas
> > > > > _______________________________________________
> > > > > Devel mailing list
> > > > > Devel(a)ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/devel
> >
> > _______________________________________________
> > Devel mailing list
> > Devel(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
2:59 a.m.
----- Original Message -----
From: "Oved Ourfali" <ovedo(a)redhat.com>
To: awels(a)redhat.com
Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
Sent: Tuesday, June 2, 2015 8:35:56 AM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
top-posting:
Due to the fact that we'll see that all over the place, I really think that
it would be best to support that at the frontend level, and not the backend
level.
well, from philosophical perspective I think that sending
org.hibernate.collection.**internal**.PersistentBag from backend to FE and than faking
it's implementation on FE
is a way to hell. It will bring lots of problems. For example the FE will be directly
dependent on the internal structure of an internal hibernate class and if we update
hibernate, FE can fail miserably.
Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems to be more tricky than
expected - it seems to be working but Omer had issues with it and in some cases (when the
delegating also retainAll()) it caused my JRE to fail on SIGSEGV.
It can be fixed on FE but it is a hack with all risks this kinds of hacks brings.
Doing it in the backend level will cause a lot of overhead.
Is it really lots of overhead? Comparing to all other layers the data has to pass anyway?
When abandoning GWT FE and start to use REST we will anyway remove the
GenericApiGWTServiceImpl which contains this cleaning
(https://gerrit.ovirt.org/#/c/41810/).
I'll leave the technical details to the UX experts here to see
what's the
right approach to do it in the frontend side.
Thanks,
Oved
----- Original Message -----
> From: "Alexander Wels" <awels(a)redhat.com>
> To: "Martin Perina" <mperina(a)redhat.com>
> Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> Sent: Tuesday, June 2, 2015 4:16:31 AM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
> On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> > ----- Original Message -----
> >
> > > From: "Alexander Wels" <awels(a)redhat.com>
> > > To: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to
> > > FE
> > >
> > > On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
> > > > ----- Original Message -----
> > > >
> > > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > > To: devel(a)ovirt.org
> > > > > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>,
"Liran Zelkha"
> > > > > <lzelkha(a)redhat.com> Sent: Monday, June 1, 2015 3:16:34
PM
> > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag sent
> > > > > to
> > > > > FE
> > > > >
> > > > > On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
> > > > > > Hey all,
> > > > > >
> > > > > > since the org.ovirt.engine.core.common.job.Job/Step... has
been
> > > > > > moved
> > > > > > to
> > > > > > use
> > > > > > the JPA we have a problem on frontend. The problem is that
the
> > > > > > @OneToMany
> > > > > > annotations results in a List which is of type
PersistentBag.
> > > > > > When
> > > > > > we
> > > > > > send
> > > > > > this to Frontend it fails during deserialization. It
actually
> > > > > > fails
> > > > > > quite
> > > > > > bad because the FE already has an ui-override of it which
is not
> > > > > > correct
> > > > > > resulting in a ton of NPEs in development mode.
> > > > > >
> > > > > > So, there are 2 nasty fixes I have made where none of them
should
> > > > > > be
> > > > > > merged
> > > > > > but demonstrate the possibilities: 1: extend the FE to be
able to
> > > > > > work
> > > > > > with
> > > > > > the PersistentBag (https://gerrit.ovirt.org/#/c/41682/)
not
> > > > > > really
> > > > > > good
> > > > > > solution since the PersistenBag is an internal Hibernate
class
> > > > > > which
> > > > > > is
> > > > > > really not meant to be passed around
> > > > > >
> > > > > > 2: fix on the backend to not send the PersistentBag but an
> > > > > > ArrayList.
> > > > > > This
> > > > > > is only a PoC fixed on a command we face the problem
> > > > > > (https://gerrit.ovirt.org/#/c/41797/) Obviously this is not
going
> > > > > > to
> > > > > > work
> > > > > > for other commands accessing the same Job nor for other
entities.
> > > > > >
> > > > > > So, the first option is generic but very very bad. The
second
> > > > > > option
> > > > > > should
> > > > > > be used but not sure how to do this in a cheep way (e.g.
without
> > > > > > using
> > > > > > reflection to deep traverse everything sent back to
frontend
> > > > > > checking
> > > > > > if
> > > > > > it
> > > > > > does not have a PersistentBag in it.
> > > > >
> > > > > Tomas,
> > > > >
> > > > > Thanks, I was investigating the same issue, I noticed it last
> > > > > Friday
> > > > > just
> > > > > before leaving, so I was investigating the problem to see what
was
> > > > > going
> > > > > on. You are right we should not be sending PersistentBag to the
> > > > > frontend
> > > > > at all. So how about we do a combination of [1] and [2], but
> > > > > instead
> > > > > of
> > > > > delegating in [1] we actually simple throw an exception stating
> > > > > don't
> > > > > sent PersistentBag to the front end. that way anyone
inadvertently
> > > > > using
> > > > > it will be notified immediately (since their code won't
work).
> > > >
> > > > Throwing the exception would help us in debugging but the main
> > > > question
> > > > is
> > > > how will we make it work? Since we are planning to move more and
more
> > > > to
> > > > JPA so we will face this issue more and more often. Solving it one
by
> > > > one
> > > > on backend in each command is not going to work.
> > >
> > > How about something like this [1]? It appears to use some aspects to
> > > translate
> > > the hibernate internal classes into normal java util classes.
> > >
> > > [1] https://code.google.com/p/dehibernator/
> >
> > Well, not sure about that from performance point of view.
> >
> > If we really move away from GWT completely in 4.0, then implementing all
> > hibernate inner collections in "uioverride" in a similar way as Tomas
did
> > seems to me as the best solution from performance and "easiest to remove
> > when not needed" point of view.
> >
>
> I took some inspiration from that code and wrote my own fairly simple
> hibernate persistent collection replacer, and I put the patch up here [3].
> It
> still uses reflection so it is probably not the fastest thing ever written.
>
> [3] https://gerrit.ovirt.org/#/c/41810/
>
> > > > > Alexander
> > > > >
> > > > > > Any better ideas?
> > > > > > Thanx,
> > > > > > Tomas
> > > > > > _______________________________________________
> > > > > > Devel mailing list
> > > > > > Devel(a)ovirt.org
> > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > >
> > > _______________________________________________
> > > Devel mailing list
> > > Devel(a)ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/devel
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
>
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
3:05 a.m.
----- Original Message -----
From: "Tomas Jelinek" <tjelinek(a)redhat.com>
To: "Oved Ourfali" <ovedo(a)redhat.com>
Cc: awels(a)redhat.com, "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
Sent: Tuesday, June 2, 2015 9:59:17 AM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
----- Original Message -----
> From: "Oved Ourfali" <ovedo(a)redhat.com>
> To: awels(a)redhat.com
> Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> Sent: Tuesday, June 2, 2015 8:35:56 AM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
> top-posting:
> Due to the fact that we'll see that all over the place, I really think that
> it would be best to support that at the frontend level, and not the backend
> level.
well, from philosophical perspective I think that sending
org.hibernate.collection.**internal**.PersistentBag from backend to FE and
than faking it's implementation on FE
is a way to hell. It will bring lots of problems. For example the FE will be
directly dependent on the internal structure of an internal hibernate class
and if we update hibernate, FE can fail miserably.
Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems to be more
tricky than expected - it seems to be working but Omer had issues with it
and in some cases (when the delegating also retainAll()) it caused my JRE to
fail on SIGSEGV.
It can be fixed on FE but it is a hack with all risks this kinds of hacks
brings.
I don't think it really matters where we do it, as long as server code will still use
the PersistentBag (otherwise we won't be able to use attached entities, and
performance will suffer).
I believe that serializing these objects to JSON/XML - will work, we won't need to do
2 level of conversions (PersistentBag-->List-->JSON/XML).
So if the issue is only on the GWT side, and we won't to write this code once, and not
per entity - where is the best way to put it?
> Doing it in the backend level will cause a lot of overhead.
Is it really lots of overhead? Comparing to all other layers the data has to
pass anyway? When abandoning GWT FE and start to use REST we will anyway
remove the GenericApiGWTServiceImpl which contains this cleaning
(https://gerrit.ovirt.org/#/c/41810/).
> I'll leave the technical details to the UX experts here to see what's the
> right approach to do it in the frontend side.
>
> Thanks,
> Oved
>
> ----- Original Message -----
> > From: "Alexander Wels" <awels(a)redhat.com>
> > To: "Martin Perina" <mperina(a)redhat.com>
> > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> > On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> > > ----- Original Message -----
> > >
> > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > To: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag
sent to
> > > > FE
> > > >
> > > > On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
> > > > > ----- Original Message -----
> > > > >
> > > > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > > > To: devel(a)ovirt.org
> > > > > > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>,
"Liran Zelkha"
> > > > > > <lzelkha(a)redhat.com> Sent: Monday, June 1, 2015
3:16:34 PM
> > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > sent
> > > > > > to
> > > > > > FE
> > > > > >
> > > > > > On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
> > > > > > > Hey all,
> > > > > > >
> > > > > > > since the org.ovirt.engine.core.common.job.Job/Step...
has been
> > > > > > > moved
> > > > > > > to
> > > > > > > use
> > > > > > > the JPA we have a problem on frontend. The problem is
that the
> > > > > > > @OneToMany
> > > > > > > annotations results in a List which is of type
PersistentBag.
> > > > > > > When
> > > > > > > we
> > > > > > > send
> > > > > > > this to Frontend it fails during deserialization. It
actually
> > > > > > > fails
> > > > > > > quite
> > > > > > > bad because the FE already has an ui-override of it
which is
> > > > > > > not
> > > > > > > correct
> > > > > > > resulting in a ton of NPEs in development mode.
> > > > > > >
> > > > > > > So, there are 2 nasty fixes I have made where none of
them
> > > > > > > should
> > > > > > > be
> > > > > > > merged
> > > > > > > but demonstrate the possibilities: 1: extend the FE to
be able
> > > > > > > to
> > > > > > > work
> > > > > > > with
> > > > > > > the PersistentBag
(https://gerrit.ovirt.org/#/c/41682/) not
> > > > > > > really
> > > > > > > good
> > > > > > > solution since the PersistenBag is an internal
Hibernate class
> > > > > > > which
> > > > > > > is
> > > > > > > really not meant to be passed around
> > > > > > >
> > > > > > > 2: fix on the backend to not send the PersistentBag
but an
> > > > > > > ArrayList.
> > > > > > > This
> > > > > > > is only a PoC fixed on a command we face the problem
> > > > > > > (https://gerrit.ovirt.org/#/c/41797/) Obviously this
is not
> > > > > > > going
> > > > > > > to
> > > > > > > work
> > > > > > > for other commands accessing the same Job nor for
other
> > > > > > > entities.
> > > > > > >
> > > > > > > So, the first option is generic but very very bad. The
second
> > > > > > > option
> > > > > > > should
> > > > > > > be used but not sure how to do this in a cheep way
(e.g.
> > > > > > > without
> > > > > > > using
> > > > > > > reflection to deep traverse everything sent back to
frontend
> > > > > > > checking
> > > > > > > if
> > > > > > > it
> > > > > > > does not have a PersistentBag in it.
> > > > > >
> > > > > > Tomas,
> > > > > >
> > > > > > Thanks, I was investigating the same issue, I noticed it
last
> > > > > > Friday
> > > > > > just
> > > > > > before leaving, so I was investigating the problem to see
what
> > > > > > was
> > > > > > going
> > > > > > on. You are right we should not be sending PersistentBag to
the
> > > > > > frontend
> > > > > > at all. So how about we do a combination of [1] and [2],
but
> > > > > > instead
> > > > > > of
> > > > > > delegating in [1] we actually simple throw an exception
stating
> > > > > > don't
> > > > > > sent PersistentBag to the front end. that way anyone
> > > > > > inadvertently
> > > > > > using
> > > > > > it will be notified immediately (since their code won't
work).
> > > > >
> > > > > Throwing the exception would help us in debugging but the main
> > > > > question
> > > > > is
> > > > > how will we make it work? Since we are planning to move more
and
> > > > > more
> > > > > to
> > > > > JPA so we will face this issue more and more often. Solving it
one
> > > > > by
> > > > > one
> > > > > on backend in each command is not going to work.
> > > >
> > > > How about something like this [1]? It appears to use some aspects to
> > > > translate
> > > > the hibernate internal classes into normal java util classes.
> > > >
> > > > [1] https://code.google.com/p/dehibernator/
> > >
> > > Well, not sure about that from performance point of view.
> > >
> > > If we really move away from GWT completely in 4.0, then implementing
> > > all
> > > hibernate inner collections in "uioverride" in a similar way as
Tomas
> > > did
> > > seems to me as the best solution from performance and "easiest to
> > > remove
> > > when not needed" point of view.
> > >
> >
> > I took some inspiration from that code and wrote my own fairly simple
> > hibernate persistent collection replacer, and I put the patch up here
> > [3].
> > It
> > still uses reflection so it is probably not the fastest thing ever
> > written.
> >
> > [3] https://gerrit.ovirt.org/#/c/41810/
> >
> > > > > > Alexander
> > > > > >
> > > > > > > Any better ideas?
> > > > > > > Thanx,
> > > > > > > Tomas
> > > > > > > _______________________________________________
> > > > > > > Devel mailing list
> > > > > > > Devel(a)ovirt.org
> > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > >
> > > > _______________________________________________
> > > > Devel mailing list
> > > > Devel(a)ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/devel
> >
> > _______________________________________________
> > Devel mailing list
> > Devel(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> >
> >
> >
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
8:51 a.m.
On Tuesday, June 02, 2015 03:05:11 AM Liran Zelkha wrote:
----- Original Message -----
> From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> To: "Oved Ourfali" <ovedo(a)redhat.com>
> Cc: awels(a)redhat.com, "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> Sent: Tuesday, June 2, 2015 9:59:17 AM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
>
>
> ----- Original Message -----
>
> > From: "Oved Ourfali" <ovedo(a)redhat.com>
> > To: awels(a)redhat.com
> > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> > Sent: Tuesday, June 2, 2015 8:35:56 AM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> > top-posting:
> > Due to the fact that we'll see that all over the place, I really think
> > that
> > it would be best to support that at the frontend level, and not the
> > backend
> > level.
>
> well, from philosophical perspective I think that sending
> org.hibernate.collection.**internal**.PersistentBag from backend to FE and
> than faking it's implementation on FE
> is a way to hell. It will bring lots of problems. For example the FE will
> be directly dependent on the internal structure of an internal hibernate
> class and if we update hibernate, FE can fail miserably.
> Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems to be more
> tricky than expected - it seems to be working but Omer had issues with it
> and in some cases (when the delegating also retainAll()) it caused my JRE
> to fail on SIGSEGV.
> It can be fixed on FE but it is a hack with all risks this kinds of hacks
> brings.
I don't think it really matters where we do it, as long as server code will
still use the PersistentBag (otherwise we won't be able to use attached
entities, and performance will suffer). I believe that serializing these
objects to JSON/XML - will work, we won't need to do 2 level of conversions
(PersistentBag-->List-->JSON/XML). So if the issue is only on the GWT side,
and we won't to write this code once, and not per entity - where is the
best way to put it?
The original problem comes from the fact that we are serializing an object
from the backend to the frontend that the frontend cannot handle (The
persistent bag). This is a backend implementation detail that the frontend
really doesn't care about. In essence GWT is Javascript that is made to look
like Java, but it has a bunch of limitations due to the fact it still ends up
as javascript in the end.
> > Doing it in the backend level will cause a lot of
overhead.
>
> Is it really lots of overhead? Comparing to all other layers the data has
> to pass anyway? When abandoning GWT FE and start to use REST we will
> anyway remove the GenericApiGWTServiceImpl which contains this cleaning
> (https://gerrit.ovirt.org/#/c/41810/).
>
The overhead comes from having to scan through every single object that we are
transferring to the front end, even though right now only one or two object
types have this particular problem. The scanning only happens if we actually
transfer the object to the frontend. If there was a way to determine if we
need to scan the objects being returned to the front end that would cut down
on the overhead. Another question to ask, is the overhead BAD enough to care?
I honestly don't know without doing some profiling. But my gut is telling me no
it is not.
> > I'll leave the technical details to the UX experts here
to see what's
> > the
> > right approach to do it in the frontend side.
> >
> > Thanks,
> > Oved
> >
> > ----- Original Message -----
> >
> > > From: "Alexander Wels" <awels(a)redhat.com>
> > > To: "Martin Perina" <mperina(a)redhat.com>
> > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to
> > > FE
> > >
> > > On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> > > > ----- Original Message -----
> > > >
> > > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > > To: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag sent
> > > > > to
> > > > > FE
> > > > >
> > > > > On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
> > > > > > ----- Original Message -----
> > > > > >
> > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > To: devel(a)ovirt.org
> > > > > > > Cc: "Tomas Jelinek"
<tjelinek(a)redhat.com>, "Liran Zelkha"
> > > > > > > <lzelkha(a)redhat.com> Sent: Monday, June 1, 2015
3:16:34 PM
> > > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > > sent
> > > > > > > to
> > > > > > > FE
> > > > > > >
> > > > > > > On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek
wrote:
> > > > > > > > Hey all,
> > > > > > > >
> > > > > > > > since the
org.ovirt.engine.core.common.job.Job/Step... has
> > > > > > > > been
> > > > > > > > moved
> > > > > > > > to
> > > > > > > > use
> > > > > > > > the JPA we have a problem on frontend. The
problem is that
> > > > > > > > the
> > > > > > > > @OneToMany
> > > > > > > > annotations results in a List which is of type
> > > > > > > > PersistentBag.
> > > > > > > > When
> > > > > > > > we
> > > > > > > > send
> > > > > > > > this to Frontend it fails during deserialization.
It
> > > > > > > > actually
> > > > > > > > fails
> > > > > > > > quite
> > > > > > > > bad because the FE already has an ui-override of
it which is
> > > > > > > > not
> > > > > > > > correct
> > > > > > > > resulting in a ton of NPEs in development mode.
> > > > > > > >
> > > > > > > > So, there are 2 nasty fixes I have made where
none of them
> > > > > > > > should
> > > > > > > > be
> > > > > > > > merged
> > > > > > > > but demonstrate the possibilities: 1: extend the
FE to be
> > > > > > > > able
> > > > > > > > to
> > > > > > > > work
> > > > > > > > with
> > > > > > > > the PersistentBag
(https://gerrit.ovirt.org/#/c/41682/) not
> > > > > > > > really
> > > > > > > > good
> > > > > > > > solution since the PersistenBag is an internal
Hibernate
> > > > > > > > class
> > > > > > > > which
> > > > > > > > is
> > > > > > > > really not meant to be passed around
> > > > > > > >
> > > > > > > > 2: fix on the backend to not send the
PersistentBag but an
> > > > > > > > ArrayList.
> > > > > > > > This
> > > > > > > > is only a PoC fixed on a command we face the
problem
> > > > > > > > (https://gerrit.ovirt.org/#/c/41797/) Obviously
this is not
> > > > > > > > going
> > > > > > > > to
> > > > > > > > work
> > > > > > > > for other commands accessing the same Job nor for
other
> > > > > > > > entities.
> > > > > > > >
> > > > > > > > So, the first option is generic but very very
bad. The
> > > > > > > > second
> > > > > > > > option
> > > > > > > > should
> > > > > > > > be used but not sure how to do this in a cheep
way (e.g.
> > > > > > > > without
> > > > > > > > using
> > > > > > > > reflection to deep traverse everything sent back
to frontend
> > > > > > > > checking
> > > > > > > > if
> > > > > > > > it
> > > > > > > > does not have a PersistentBag in it.
> > > > > > >
> > > > > > > Tomas,
> > > > > > >
> > > > > > > Thanks, I was investigating the same issue, I noticed
it last
> > > > > > > Friday
> > > > > > > just
> > > > > > > before leaving, so I was investigating the problem to
see what
> > > > > > > was
> > > > > > > going
> > > > > > > on. You are right we should not be sending
PersistentBag to
> > > > > > > the
> > > > > > > frontend
> > > > > > > at all. So how about we do a combination of [1] and
[2], but
> > > > > > > instead
> > > > > > > of
> > > > > > > delegating in [1] we actually simple throw an
exception
> > > > > > > stating
> > > > > > > don't
> > > > > > > sent PersistentBag to the front end. that way anyone
> > > > > > > inadvertently
> > > > > > > using
> > > > > > > it will be notified immediately (since their code
won't work).
> > > > > >
> > > > > > Throwing the exception would help us in debugging but the
main
> > > > > > question
> > > > > > is
> > > > > > how will we make it work? Since we are planning to move
more and
> > > > > > more
> > > > > > to
> > > > > > JPA so we will face this issue more and more often. Solving
it
> > > > > > one
> > > > > > by
> > > > > > one
> > > > > > on backend in each command is not going to work.
> > > > >
> > > > > How about something like this [1]? It appears to use some
aspects
> > > > > to
> > > > > translate
> > > > > the hibernate internal classes into normal java util classes.
> > > > >
> > > > > [1] https://code.google.com/p/dehibernator/
> > > >
> > > > Well, not sure about that from performance point of view.
> > > >
> > > > If we really move away from GWT completely in 4.0, then implementing
> > > > all
> > > > hibernate inner collections in "uioverride" in a similar
way as
> > > > Tomas
> > > > did
> > > > seems to me as the best solution from performance and "easiest
to
> > > > remove
> > > > when not needed" point of view.
> > >
> > > I took some inspiration from that code and wrote my own fairly simple
> > > hibernate persistent collection replacer, and I put the patch up here
> > > [3].
> > > It
> > > still uses reflection so it is probably not the fastest thing ever
> > > written.
> > >
> > > [3] https://gerrit.ovirt.org/#/c/41810/
> > >
> > > > > > > Alexander
> > > > > > >
> > > > > > > > Any better ideas?
> > > > > > > > Thanx,
> > > > > > > > Tomas
> > > > > > > > _______________________________________________
> > > > > > > > Devel mailing list
> > > > > > > > Devel(a)ovirt.org
> > > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > >
> > > > > _______________________________________________
> > > > > Devel mailing list
> > > > > Devel(a)ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > >
> > > _______________________________________________
> > > Devel mailing list
> > > Devel(a)ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/devel
> >
> > _______________________________________________
> > Devel mailing list
> > Devel(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
9:08 a.m.
----- Original Message -----
From: "Alexander Wels" <awels(a)redhat.com>
To: "Liran Zelkha" <lzelkha(a)redhat.com>
Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Oved Ourfali"
<ovedo(a)redhat.com>, devel(a)ovirt.org
Sent: Tuesday, June 2, 2015 2:51:31 PM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
On Tuesday, June 02, 2015 03:05:11 AM Liran Zelkha wrote:
> ----- Original Message -----
>
> > From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > To: "Oved Ourfali" <ovedo(a)redhat.com>
> > Cc: awels(a)redhat.com, "Liran Zelkha" <lzelkha(a)redhat.com>,
> > devel(a)ovirt.org
> > Sent: Tuesday, June 2, 2015 9:59:17 AM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> >
> >
> > ----- Original Message -----
> >
> > > From: "Oved Ourfali" <ovedo(a)redhat.com>
> > > To: awels(a)redhat.com
> > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > Sent: Tuesday, June 2, 2015 8:35:56 AM
> > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to
> > > FE
> > >
> > > top-posting:
> > > Due to the fact that we'll see that all over the place, I really
think
> > > that
> > > it would be best to support that at the frontend level, and not the
> > > backend
> > > level.
> >
> > well, from philosophical perspective I think that sending
> > org.hibernate.collection.**internal**.PersistentBag from backend to FE
> > and
> > than faking it's implementation on FE
> > is a way to hell. It will bring lots of problems. For example the FE will
> > be directly dependent on the internal structure of an internal hibernate
> > class and if we update hibernate, FE can fail miserably.
> > Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems to be more
> > tricky than expected - it seems to be working but Omer had issues with it
> > and in some cases (when the delegating also retainAll()) it caused my JRE
> > to fail on SIGSEGV.
> > It can be fixed on FE but it is a hack with all risks this kinds of hacks
> > brings.
>
> I don't think it really matters where we do it, as long as server code will
> still use the PersistentBag (otherwise we won't be able to use attached
> entities, and performance will suffer). I believe that serializing these
> objects to JSON/XML - will work, we won't need to do 2 level of conversions
> (PersistentBag-->List-->JSON/XML). So if the issue is only on the GWT side,
> and we won't to write this code once, and not per entity - where is the
> best way to put it?
The original problem comes from the fact that we are serializing an object
from the backend to the frontend that the frontend cannot handle (The
persistent bag). This is a backend implementation detail that the frontend
really doesn't care about. In essence GWT is Javascript that is made to look
like Java, but it has a bunch of limitations due to the fact it still ends up
as javascript in the end.
> > > Doing it in the backend level will cause a lot of overhead.
> >
> > Is it really lots of overhead? Comparing to all other layers the data has
> > to pass anyway? When abandoning GWT FE and start to use REST we will
> > anyway remove the GenericApiGWTServiceImpl which contains this cleaning
> > (https://gerrit.ovirt.org/#/c/41810/).
> >
The overhead comes from having to scan through every single object that we
are
transferring to the front end, even though right now only one or two object
types have this particular problem. The scanning only happens if we actually
transfer the object to the frontend. If there was a way to determine if we
need to scan the objects being returned to the front end that would cut down
on the overhead. Another question to ask, is the overhead BAD enough to care?
I honestly don't know without doing some profiling. But my gut is telling me
no
it is not.
I agree this is the correct fix on the correct place. On the other hand, it needs some
optimizations and tests if it actually works and if it is not a big bottleneck.
Also we need to consider if we want to have some annotation on classes we want to filter
before sending back to frontend or we want this to be done all the time.
Long story short, I think the patch which broke the frontend debugging should be reverted
and merged once the https://gerrit.ovirt.org/#/c/41810/ is finished and tested.
@Liran: could you please revert your patch? Because currently the FE debugging is pretty
much broken (and I don't trust the web mode either...) and the solution is not
completely clear now.
> > > I'll leave the technical details to the UX experts here to see
what's
> > > the
> > > right approach to do it in the frontend side.
> > >
> > > Thanks,
> > > Oved
> > >
> > > ----- Original Message -----
> > >
> > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > To: "Martin Perina" <mperina(a)redhat.com>
> > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag
sent to
> > > > FE
> > > >
> > > > On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> > > > > ----- Original Message -----
> > > > >
> > > > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > > > To: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > sent
> > > > > > to
> > > > > > FE
> > > > > >
> > > > > > On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
> > > > > > > ----- Original Message -----
> > > > > > >
> > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > To: devel(a)ovirt.org
> > > > > > > > Cc: "Tomas Jelinek"
<tjelinek(a)redhat.com>, "Liran Zelkha"
> > > > > > > > <lzelkha(a)redhat.com> Sent: Monday, June 1,
2015 3:16:34 PM
> > > > > > > > Subject: Re: [ovirt-devel] hibernate's
internal PersistentBag
> > > > > > > > sent
> > > > > > > > to
> > > > > > > > FE
> > > > > > > >
> > > > > > > > On Monday, June 01, 2015 09:08:50 AM Tomas
Jelinek wrote:
> > > > > > > > > Hey all,
> > > > > > > > >
> > > > > > > > > since the
org.ovirt.engine.core.common.job.Job/Step... has
> > > > > > > > > been
> > > > > > > > > moved
> > > > > > > > > to
> > > > > > > > > use
> > > > > > > > > the JPA we have a problem on frontend. The
problem is that
> > > > > > > > > the
> > > > > > > > > @OneToMany
> > > > > > > > > annotations results in a List which is of
type
> > > > > > > > > PersistentBag.
> > > > > > > > > When
> > > > > > > > > we
> > > > > > > > > send
> > > > > > > > > this to Frontend it fails during
deserialization. It
> > > > > > > > > actually
> > > > > > > > > fails
> > > > > > > > > quite
> > > > > > > > > bad because the FE already has an
ui-override of it which
> > > > > > > > > is
> > > > > > > > > not
> > > > > > > > > correct
> > > > > > > > > resulting in a ton of NPEs in development
mode.
> > > > > > > > >
> > > > > > > > > So, there are 2 nasty fixes I have made
where none of them
> > > > > > > > > should
> > > > > > > > > be
> > > > > > > > > merged
> > > > > > > > > but demonstrate the possibilities: 1: extend
the FE to be
> > > > > > > > > able
> > > > > > > > > to
> > > > > > > > > work
> > > > > > > > > with
> > > > > > > > > the PersistentBag
(https://gerrit.ovirt.org/#/c/41682/) not
> > > > > > > > > really
> > > > > > > > > good
> > > > > > > > > solution since the PersistenBag is an
internal Hibernate
> > > > > > > > > class
> > > > > > > > > which
> > > > > > > > > is
> > > > > > > > > really not meant to be passed around
> > > > > > > > >
> > > > > > > > > 2: fix on the backend to not send the
PersistentBag but an
> > > > > > > > > ArrayList.
> > > > > > > > > This
> > > > > > > > > is only a PoC fixed on a command we face the
problem
> > > > > > > > > (https://gerrit.ovirt.org/#/c/41797/)
Obviously this is not
> > > > > > > > > going
> > > > > > > > > to
> > > > > > > > > work
> > > > > > > > > for other commands accessing the same Job
nor for other
> > > > > > > > > entities.
> > > > > > > > >
> > > > > > > > > So, the first option is generic but very
very bad. The
> > > > > > > > > second
> > > > > > > > > option
> > > > > > > > > should
> > > > > > > > > be used but not sure how to do this in a
cheep way (e.g.
> > > > > > > > > without
> > > > > > > > > using
> > > > > > > > > reflection to deep traverse everything sent
back to
> > > > > > > > > frontend
> > > > > > > > > checking
> > > > > > > > > if
> > > > > > > > > it
> > > > > > > > > does not have a PersistentBag in it.
> > > > > > > >
> > > > > > > > Tomas,
> > > > > > > >
> > > > > > > > Thanks, I was investigating the same issue, I
noticed it last
> > > > > > > > Friday
> > > > > > > > just
> > > > > > > > before leaving, so I was investigating the
problem to see
> > > > > > > > what
> > > > > > > > was
> > > > > > > > going
> > > > > > > > on. You are right we should not be sending
PersistentBag to
> > > > > > > > the
> > > > > > > > frontend
> > > > > > > > at all. So how about we do a combination of [1]
and [2], but
> > > > > > > > instead
> > > > > > > > of
> > > > > > > > delegating in [1] we actually simple throw an
exception
> > > > > > > > stating
> > > > > > > > don't
> > > > > > > > sent PersistentBag to the front end. that way
anyone
> > > > > > > > inadvertently
> > > > > > > > using
> > > > > > > > it will be notified immediately (since their code
won't
> > > > > > > > work).
> > > > > > >
> > > > > > > Throwing the exception would help us in debugging but
the main
> > > > > > > question
> > > > > > > is
> > > > > > > how will we make it work? Since we are planning to
move more
> > > > > > > and
> > > > > > > more
> > > > > > > to
> > > > > > > JPA so we will face this issue more and more often.
Solving it
> > > > > > > one
> > > > > > > by
> > > > > > > one
> > > > > > > on backend in each command is not going to work.
> > > > > >
> > > > > > How about something like this [1]? It appears to use some
aspects
> > > > > > to
> > > > > > translate
> > > > > > the hibernate internal classes into normal java util
classes.
> > > > > >
> > > > > > [1] https://code.google.com/p/dehibernator/
> > > > >
> > > > > Well, not sure about that from performance point of view.
> > > > >
> > > > > If we really move away from GWT completely in 4.0, then
> > > > > implementing
> > > > > all
> > > > > hibernate inner collections in "uioverride" in a
similar way as
> > > > > Tomas
> > > > > did
> > > > > seems to me as the best solution from performance and
"easiest to
> > > > > remove
> > > > > when not needed" point of view.
> > > >
> > > > I took some inspiration from that code and wrote my own fairly
simple
> > > > hibernate persistent collection replacer, and I put the patch up
here
> > > > [3].
> > > > It
> > > > still uses reflection so it is probably not the fastest thing ever
> > > > written.
> > > >
> > > > [3] https://gerrit.ovirt.org/#/c/41810/
> > > >
> > > > > > > > Alexander
> > > > > > > >
> > > > > > > > > Any better ideas?
> > > > > > > > > Thanx,
> > > > > > > > > Tomas
> > > > > > > > >
_______________________________________________
> > > > > > > > > Devel mailing list
> > > > > > > > > Devel(a)ovirt.org
> > > > > > > > >
http://lists.ovirt.org/mailman/listinfo/devel
> > > > > >
> > > > > > _______________________________________________
> > > > > > Devel mailing list
> > > > > > Devel(a)ovirt.org
> > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > >
> > > > _______________________________________________
> > > > Devel mailing list
> > > > Devel(a)ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/devel
> > >
> > > _______________________________________________
> > > Devel mailing list
> > > Devel(a)ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/devel
9:10 a.m.
----- Original Message -----
From: "Tomas Jelinek" <tjelinek(a)redhat.com>
To: awels(a)redhat.com
Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, "Oved Ourfali"
<ovedo(a)redhat.com>, devel(a)ovirt.org
Sent: Tuesday, June 2, 2015 4:08:05 PM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
----- Original Message -----
> From: "Alexander Wels" <awels(a)redhat.com>
> To: "Liran Zelkha" <lzelkha(a)redhat.com>
> Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Oved Ourfali"
> <ovedo(a)redhat.com>, devel(a)ovirt.org
> Sent: Tuesday, June 2, 2015 2:51:31 PM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
> On Tuesday, June 02, 2015 03:05:11 AM Liran Zelkha wrote:
> > ----- Original Message -----
> >
> > > From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > To: "Oved Ourfali" <ovedo(a)redhat.com>
> > > Cc: awels(a)redhat.com, "Liran Zelkha"
<lzelkha(a)redhat.com>,
> > > devel(a)ovirt.org
> > > Sent: Tuesday, June 2, 2015 9:59:17 AM
> > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to
> > > FE
> > >
> > >
> > >
> > > ----- Original Message -----
> > >
> > > > From: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > To: awels(a)redhat.com
> > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > Sent: Tuesday, June 2, 2015 8:35:56 AM
> > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag
sent to
> > > > FE
> > > >
> > > > top-posting:
> > > > Due to the fact that we'll see that all over the place, I really
> > > > think
> > > > that
> > > > it would be best to support that at the frontend level, and not the
> > > > backend
> > > > level.
> > >
> > > well, from philosophical perspective I think that sending
> > > org.hibernate.collection.**internal**.PersistentBag from backend to FE
> > > and
> > > than faking it's implementation on FE
> > > is a way to hell. It will bring lots of problems. For example the FE
> > > will
> > > be directly dependent on the internal structure of an internal
> > > hibernate
> > > class and if we update hibernate, FE can fail miserably.
> > > Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems to be
> > > more
> > > tricky than expected - it seems to be working but Omer had issues with
> > > it
> > > and in some cases (when the delegating also retainAll()) it caused my
> > > JRE
> > > to fail on SIGSEGV.
> > > It can be fixed on FE but it is a hack with all risks this kinds of
> > > hacks
> > > brings.
> >
> > I don't think it really matters where we do it, as long as server code
> > will
> > still use the PersistentBag (otherwise we won't be able to use attached
> > entities, and performance will suffer). I believe that serializing these
> > objects to JSON/XML - will work, we won't need to do 2 level of
> > conversions
> > (PersistentBag-->List-->JSON/XML). So if the issue is only on the GWT
> > side,
> > and we won't to write this code once, and not per entity - where is the
> > best way to put it?
>
> The original problem comes from the fact that we are serializing an object
> from the backend to the frontend that the frontend cannot handle (The
> persistent bag). This is a backend implementation detail that the frontend
> really doesn't care about. In essence GWT is Javascript that is made to
> look
> like Java, but it has a bunch of limitations due to the fact it still ends
> up
> as javascript in the end.
>
> > > > Doing it in the backend level will cause a lot of overhead.
> > >
> > > Is it really lots of overhead? Comparing to all other layers the data
> > > has
> > > to pass anyway? When abandoning GWT FE and start to use REST we will
> > > anyway remove the GenericApiGWTServiceImpl which contains this cleaning
> > > (https://gerrit.ovirt.org/#/c/41810/).
> > >
>
> The overhead comes from having to scan through every single object that we
> are
> transferring to the front end, even though right now only one or two object
> types have this particular problem. The scanning only happens if we
> actually
> transfer the object to the frontend. If there was a way to determine if we
> need to scan the objects being returned to the front end that would cut
> down
> on the overhead. Another question to ask, is the overhead BAD enough to
> care?
> I honestly don't know without doing some profiling. But my gut is telling
> me
> no
> it is not.
I agree this is the correct fix on the correct place. On the other hand, it
needs some optimizations and tests if it actually works and if it is not a
big bottleneck.
Also we need to consider if we want to have some annotation on classes we
want to filter before sending back to frontend or we want this to be done
all the time.
Long story short, I think the patch which broke the frontend debugging should
be reverted and merged once the https://gerrit.ovirt.org/#/c/41810/ is
finished and tested.
What's the ETA on finishing https://gerrit.ovirt.org/#/c/41810 ?
@Liran: could you please revert your patch? Because currently the FE
debugging is pretty much broken (and I don't trust the web mode either...)
and the solution is not completely clear now.
Let's see what the ETA is, as perhaps we can merge something soon.
Liran's patch is a big change in all DB layers, so won't be easy to revert.
>
> > > > I'll leave the technical details to the UX experts here to see
what's
> > > > the
> > > > right approach to do it in the frontend side.
> > > >
> > > > Thanks,
> > > > Oved
> > > >
> > > > ----- Original Message -----
> > > >
> > > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > > To: "Martin Perina" <mperina(a)redhat.com>
> > > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag sent
> > > > > to
> > > > > FE
> > > > >
> > > > > On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> > > > > > ----- Original Message -----
> > > > > >
> > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > To: "Tomas Jelinek"
<tjelinek(a)redhat.com>
> > > > > > > Cc: "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > > > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > > sent
> > > > > > > to
> > > > > > > FE
> > > > > > >
> > > > > > > On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek
wrote:
> > > > > > > > ----- Original Message -----
> > > > > > > >
> > > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > > To: devel(a)ovirt.org
> > > > > > > > > Cc: "Tomas Jelinek"
<tjelinek(a)redhat.com>, "Liran Zelkha"
> > > > > > > > > <lzelkha(a)redhat.com> Sent: Monday,
June 1, 2015 3:16:34 PM
> > > > > > > > > Subject: Re: [ovirt-devel] hibernate's
internal
> > > > > > > > > PersistentBag
> > > > > > > > > sent
> > > > > > > > > to
> > > > > > > > > FE
> > > > > > > > >
> > > > > > > > > On Monday, June 01, 2015 09:08:50 AM Tomas
Jelinek wrote:
> > > > > > > > > > Hey all,
> > > > > > > > > >
> > > > > > > > > > since the
org.ovirt.engine.core.common.job.Job/Step...
> > > > > > > > > > has
> > > > > > > > > > been
> > > > > > > > > > moved
> > > > > > > > > > to
> > > > > > > > > > use
> > > > > > > > > > the JPA we have a problem on frontend.
The problem is
> > > > > > > > > > that
> > > > > > > > > > the
> > > > > > > > > > @OneToMany
> > > > > > > > > > annotations results in a List which is
of type
> > > > > > > > > > PersistentBag.
> > > > > > > > > > When
> > > > > > > > > > we
> > > > > > > > > > send
> > > > > > > > > > this to Frontend it fails during
deserialization. It
> > > > > > > > > > actually
> > > > > > > > > > fails
> > > > > > > > > > quite
> > > > > > > > > > bad because the FE already has an
ui-override of it which
> > > > > > > > > > is
> > > > > > > > > > not
> > > > > > > > > > correct
> > > > > > > > > > resulting in a ton of NPEs in
development mode.
> > > > > > > > > >
> > > > > > > > > > So, there are 2 nasty fixes I have made
where none of
> > > > > > > > > > them
> > > > > > > > > > should
> > > > > > > > > > be
> > > > > > > > > > merged
> > > > > > > > > > but demonstrate the possibilities: 1:
extend the FE to be
> > > > > > > > > > able
> > > > > > > > > > to
> > > > > > > > > > work
> > > > > > > > > > with
> > > > > > > > > > the PersistentBag
(https://gerrit.ovirt.org/#/c/41682/)
> > > > > > > > > > not
> > > > > > > > > > really
> > > > > > > > > > good
> > > > > > > > > > solution since the PersistenBag is an
internal Hibernate
> > > > > > > > > > class
> > > > > > > > > > which
> > > > > > > > > > is
> > > > > > > > > > really not meant to be passed around
> > > > > > > > > >
> > > > > > > > > > 2: fix on the backend to not send the
PersistentBag but
> > > > > > > > > > an
> > > > > > > > > > ArrayList.
> > > > > > > > > > This
> > > > > > > > > > is only a PoC fixed on a command we
face the problem
> > > > > > > > > > (https://gerrit.ovirt.org/#/c/41797/)
Obviously this is
> > > > > > > > > > not
> > > > > > > > > > going
> > > > > > > > > > to
> > > > > > > > > > work
> > > > > > > > > > for other commands accessing the same
Job nor for other
> > > > > > > > > > entities.
> > > > > > > > > >
> > > > > > > > > > So, the first option is generic but
very very bad. The
> > > > > > > > > > second
> > > > > > > > > > option
> > > > > > > > > > should
> > > > > > > > > > be used but not sure how to do this in
a cheep way (e.g.
> > > > > > > > > > without
> > > > > > > > > > using
> > > > > > > > > > reflection to deep traverse everything
sent back to
> > > > > > > > > > frontend
> > > > > > > > > > checking
> > > > > > > > > > if
> > > > > > > > > > it
> > > > > > > > > > does not have a PersistentBag in it.
> > > > > > > > >
> > > > > > > > > Tomas,
> > > > > > > > >
> > > > > > > > > Thanks, I was investigating the same issue,
I noticed it
> > > > > > > > > last
> > > > > > > > > Friday
> > > > > > > > > just
> > > > > > > > > before leaving, so I was investigating the
problem to see
> > > > > > > > > what
> > > > > > > > > was
> > > > > > > > > going
> > > > > > > > > on. You are right we should not be sending
PersistentBag to
> > > > > > > > > the
> > > > > > > > > frontend
> > > > > > > > > at all. So how about we do a combination of
[1] and [2],
> > > > > > > > > but
> > > > > > > > > instead
> > > > > > > > > of
> > > > > > > > > delegating in [1] we actually simple throw
an exception
> > > > > > > > > stating
> > > > > > > > > don't
> > > > > > > > > sent PersistentBag to the front end. that
way anyone
> > > > > > > > > inadvertently
> > > > > > > > > using
> > > > > > > > > it will be notified immediately (since their
code won't
> > > > > > > > > work).
> > > > > > > >
> > > > > > > > Throwing the exception would help us in debugging
but the
> > > > > > > > main
> > > > > > > > question
> > > > > > > > is
> > > > > > > > how will we make it work? Since we are planning
to move more
> > > > > > > > and
> > > > > > > > more
> > > > > > > > to
> > > > > > > > JPA so we will face this issue more and more
often. Solving
> > > > > > > > it
> > > > > > > > one
> > > > > > > > by
> > > > > > > > one
> > > > > > > > on backend in each command is not going to work.
> > > > > > >
> > > > > > > How about something like this [1]? It appears to use
some
> > > > > > > aspects
> > > > > > > to
> > > > > > > translate
> > > > > > > the hibernate internal classes into normal java util
classes.
> > > > > > >
> > > > > > > [1] https://code.google.com/p/dehibernator/
> > > > > >
> > > > > > Well, not sure about that from performance point of view.
> > > > > >
> > > > > > If we really move away from GWT completely in 4.0, then
> > > > > > implementing
> > > > > > all
> > > > > > hibernate inner collections in "uioverride" in a
similar way as
> > > > > > Tomas
> > > > > > did
> > > > > > seems to me as the best solution from performance and
"easiest to
> > > > > > remove
> > > > > > when not needed" point of view.
> > > > >
> > > > > I took some inspiration from that code and wrote my own fairly
> > > > > simple
> > > > > hibernate persistent collection replacer, and I put the patch
up
> > > > > here
> > > > > [3].
> > > > > It
> > > > > still uses reflection so it is probably not the fastest thing
ever
> > > > > written.
> > > > >
> > > > > [3] https://gerrit.ovirt.org/#/c/41810/
> > > > >
> > > > > > > > > Alexander
> > > > > > > > >
> > > > > > > > > > Any better ideas?
> > > > > > > > > > Thanx,
> > > > > > > > > > Tomas
> > > > > > > > > >
_______________________________________________
> > > > > > > > > > Devel mailing list
> > > > > > > > > > Devel(a)ovirt.org
> > > > > > > > > >
http://lists.ovirt.org/mailman/listinfo/devel
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > Devel mailing list
> > > > > > > Devel(a)ovirt.org
> > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > >
> > > > > _______________________________________________
> > > > > Devel mailing list
> > > > > Devel(a)ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > >
> > > > _______________________________________________
> > > > Devel mailing list
> > > > Devel(a)ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/devel
>
>
9:19 a.m.
On Tue, Jun 2, 2015 at 3:10 PM, Oved Ourfali <ovedo(a)redhat.com> wrote:
----- Original Message -----
> From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> To: awels(a)redhat.com
> Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, "Oved Ourfali"
<ovedo(a)redhat.com>, devel(a)ovirt.org
> Sent: Tuesday, June 2, 2015 4:08:05 PM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
>
>
> ----- Original Message -----
> > From: "Alexander Wels" <awels(a)redhat.com>
> > To: "Liran Zelkha" <lzelkha(a)redhat.com>
> > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Oved
Ourfali"
> > <ovedo(a)redhat.com>, devel(a)ovirt.org
> > Sent: Tuesday, June 2, 2015 2:51:31 PM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> > On Tuesday, June 02, 2015 03:05:11 AM Liran Zelkha wrote:
> > > ----- Original Message -----
> > >
> > > > From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > > To: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > Cc: awels(a)redhat.com, "Liran Zelkha"
<lzelkha(a)redhat.com>,
> > > > devel(a)ovirt.org
> > > > Sent: Tuesday, June 2, 2015 9:59:17 AM
> > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent
to
> > > > FE
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > >
> > > > > From: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > > To: awels(a)redhat.com
> > > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > > Sent: Tuesday, June 2, 2015 8:35:56 AM
> > > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag
sent to
> > > > > FE
> > > > >
> > > > > top-posting:
> > > > > Due to the fact that we'll see that all over the place, I
really
> > > > > think
> > > > > that
> > > > > it would be best to support that at the frontend level, and not
the
> > > > > backend
> > > > > level.
> > > >
> > > > well, from philosophical perspective I think that sending
> > > > org.hibernate.collection.**internal**.PersistentBag from backend to
FE
> > > > and
> > > > than faking it's implementation on FE
> > > > is a way to hell. It will bring lots of problems. For example the FE
> > > > will
> > > > be directly dependent on the internal structure of an internal
> > > > hibernate
> > > > class and if we update hibernate, FE can fail miserably.
> > > > Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems to be
> > > > more
> > > > tricky than expected - it seems to be working but Omer had issues
with
> > > > it
> > > > and in some cases (when the delegating also retainAll()) it caused my
> > > > JRE
> > > > to fail on SIGSEGV.
> > > > It can be fixed on FE but it is a hack with all risks this kinds of
> > > > hacks
> > > > brings.
> > >
> > > I don't think it really matters where we do it, as long as server code
> > > will
> > > still use the PersistentBag (otherwise we won't be able to use
attached
> > > entities, and performance will suffer). I believe that serializing these
> > > objects to JSON/XML - will work, we won't need to do 2 level of
> > > conversions
> > > (PersistentBag-->List-->JSON/XML). So if the issue is only on the
GWT
> > > side,
> > > and we won't to write this code once, and not per entity - where is
the
> > > best way to put it?
> >
> > The original problem comes from the fact that we are serializing an object
> > from the backend to the frontend that the frontend cannot handle (The
> > persistent bag). This is a backend implementation detail that the frontend
> > really doesn't care about. In essence GWT is Javascript that is made to
> > look
> > like Java, but it has a bunch of limitations due to the fact it still ends
> > up
> > as javascript in the end.
> >
> > > > > Doing it in the backend level will cause a lot of overhead.
> > > >
> > > > Is it really lots of overhead? Comparing to all other layers the data
> > > > has
> > > > to pass anyway? When abandoning GWT FE and start to use REST we will
> > > > anyway remove the GenericApiGWTServiceImpl which contains this
cleaning
> > > > (https://gerrit.ovirt.org/#/c/41810/).
> > > >
> >
> > The overhead comes from having to scan through every single object that we
> > are
> > transferring to the front end, even though right now only one or two object
> > types have this particular problem. The scanning only happens if we
> > actually
> > transfer the object to the frontend. If there was a way to determine if we
> > need to scan the objects being returned to the front end that would cut
> > down
> > on the overhead. Another question to ask, is the overhead BAD enough to
> > care?
> > I honestly don't know without doing some profiling. But my gut is telling
> > me
> > no
> > it is not.
>
> I agree this is the correct fix on the correct place. On the other hand, it
> needs some optimizations and tests if it actually works and if it is not a
> big bottleneck.
> Also we need to consider if we want to have some annotation on classes we
> want to filter before sending back to frontend or we want this to be done
> all the time.
>
> Long story short, I think the patch which broke the frontend debugging should
> be reverted and merged once the https://gerrit.ovirt.org/#/c/41810/ is
> finished and tested.
What's the ETA on finishing https://gerrit.ovirt.org/#/c/41810 ?
> @Liran: could you please revert your patch? Because currently the FE
> debugging is pretty much broken (and I don't trust the web mode either...)
> and the solution is not completely clear now.
>
Let's see what the ETA is, as perhaps we can merge something soon.
Liran's patch is a big change in all DB layers, so won't be easy to revert.
I suggest to use DTO [1] concept to solve this issue. We should not marshal
entities and send them anywhere.
[1] http://en.wikipedia.org/wiki/Data_transfer_object
>
> >
> > > > > I'll leave the technical details to the UX experts here to
see what's
> > > > > the
> > > > > right approach to do it in the frontend side.
> > > > >
> > > > > Thanks,
> > > > > Oved
> > > > >
> > > > > ----- Original Message -----
> > > > >
> > > > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > > > To: "Martin Perina" <mperina(a)redhat.com>
> > > > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > > > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag sent
> > > > > > to
> > > > > > FE
> > > > > >
> > > > > > On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> > > > > > > ----- Original Message -----
> > > > > > >
> > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > To: "Tomas Jelinek"
<tjelinek(a)redhat.com>
> > > > > > > > Cc: "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > > > > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > > > > > Subject: Re: [ovirt-devel] hibernate's
internal PersistentBag
> > > > > > > > sent
> > > > > > > > to
> > > > > > > > FE
> > > > > > > >
> > > > > > > > On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek
wrote:
> > > > > > > > > ----- Original Message -----
> > > > > > > > >
> > > > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > > > To: devel(a)ovirt.org
> > > > > > > > > > Cc: "Tomas Jelinek"
<tjelinek(a)redhat.com>, "Liran Zelkha"
> > > > > > > > > > <lzelkha(a)redhat.com> Sent: Monday,
June 1, 2015 3:16:34 PM
> > > > > > > > > > Subject: Re: [ovirt-devel]
hibernate's internal
> > > > > > > > > > PersistentBag
> > > > > > > > > > sent
> > > > > > > > > > to
> > > > > > > > > > FE
> > > > > > > > > >
> > > > > > > > > > On Monday, June 01, 2015 09:08:50 AM
Tomas Jelinek wrote:
> > > > > > > > > > > Hey all,
> > > > > > > > > > >
> > > > > > > > > > > since the
org.ovirt.engine.core.common.job.Job/Step...
> > > > > > > > > > > has
> > > > > > > > > > > been
> > > > > > > > > > > moved
> > > > > > > > > > > to
> > > > > > > > > > > use
> > > > > > > > > > > the JPA we have a problem on
frontend. The problem is
> > > > > > > > > > > that
> > > > > > > > > > > the
> > > > > > > > > > > @OneToMany
> > > > > > > > > > > annotations results in a List which
is of type
> > > > > > > > > > > PersistentBag.
> > > > > > > > > > > When
> > > > > > > > > > > we
> > > > > > > > > > > send
> > > > > > > > > > > this to Frontend it fails during
deserialization. It
> > > > > > > > > > > actually
> > > > > > > > > > > fails
> > > > > > > > > > > quite
> > > > > > > > > > > bad because the FE already has an
ui-override of it which
> > > > > > > > > > > is
> > > > > > > > > > > not
> > > > > > > > > > > correct
> > > > > > > > > > > resulting in a ton of NPEs in
development mode.
> > > > > > > > > > >
> > > > > > > > > > > So, there are 2 nasty fixes I have
made where none of
> > > > > > > > > > > them
> > > > > > > > > > > should
> > > > > > > > > > > be
> > > > > > > > > > > merged
> > > > > > > > > > > but demonstrate the possibilities:
1: extend the FE to be
> > > > > > > > > > > able
> > > > > > > > > > > to
> > > > > > > > > > > work
> > > > > > > > > > > with
> > > > > > > > > > > the PersistentBag
(https://gerrit.ovirt.org/#/c/41682/)
> > > > > > > > > > > not
> > > > > > > > > > > really
> > > > > > > > > > > good
> > > > > > > > > > > solution since the PersistenBag is
an internal Hibernate
> > > > > > > > > > > class
> > > > > > > > > > > which
> > > > > > > > > > > is
> > > > > > > > > > > really not meant to be passed
around
> > > > > > > > > > >
> > > > > > > > > > > 2: fix on the backend to not send
the PersistentBag but
> > > > > > > > > > > an
> > > > > > > > > > > ArrayList.
> > > > > > > > > > > This
> > > > > > > > > > > is only a PoC fixed on a command we
face the problem
> > > > > > > > > > >
(https://gerrit.ovirt.org/#/c/41797/) Obviously this is
> > > > > > > > > > > not
> > > > > > > > > > > going
> > > > > > > > > > > to
> > > > > > > > > > > work
> > > > > > > > > > > for other commands accessing the
same Job nor for other
> > > > > > > > > > > entities.
> > > > > > > > > > >
> > > > > > > > > > > So, the first option is generic but
very very bad. The
> > > > > > > > > > > second
> > > > > > > > > > > option
> > > > > > > > > > > should
> > > > > > > > > > > be used but not sure how to do this
in a cheep way (e.g.
> > > > > > > > > > > without
> > > > > > > > > > > using
> > > > > > > > > > > reflection to deep traverse
everything sent back to
> > > > > > > > > > > frontend
> > > > > > > > > > > checking
> > > > > > > > > > > if
> > > > > > > > > > > it
> > > > > > > > > > > does not have a PersistentBag in
it.
> > > > > > > > > >
> > > > > > > > > > Tomas,
> > > > > > > > > >
> > > > > > > > > > Thanks, I was investigating the same
issue, I noticed it
> > > > > > > > > > last
> > > > > > > > > > Friday
> > > > > > > > > > just
> > > > > > > > > > before leaving, so I was investigating
the problem to see
> > > > > > > > > > what
> > > > > > > > > > was
> > > > > > > > > > going
> > > > > > > > > > on. You are right we should not be
sending PersistentBag to
> > > > > > > > > > the
> > > > > > > > > > frontend
> > > > > > > > > > at all. So how about we do a combination
of [1] and [2],
> > > > > > > > > > but
> > > > > > > > > > instead
> > > > > > > > > > of
> > > > > > > > > > delegating in [1] we actually simple
throw an exception
> > > > > > > > > > stating
> > > > > > > > > > don't
> > > > > > > > > > sent PersistentBag to the front end.
that way anyone
> > > > > > > > > > inadvertently
> > > > > > > > > > using
> > > > > > > > > > it will be notified immediately (since
their code won't
> > > > > > > > > > work).
> > > > > > > > >
> > > > > > > > > Throwing the exception would help us in
debugging but the
> > > > > > > > > main
> > > > > > > > > question
> > > > > > > > > is
> > > > > > > > > how will we make it work? Since we are
planning to move more
> > > > > > > > > and
> > > > > > > > > more
> > > > > > > > > to
> > > > > > > > > JPA so we will face this issue more and more
often. Solving
> > > > > > > > > it
> > > > > > > > > one
> > > > > > > > > by
> > > > > > > > > one
> > > > > > > > > on backend in each command is not going to
work.
> > > > > > > >
> > > > > > > > How about something like this [1]? It appears to
use some
> > > > > > > > aspects
> > > > > > > > to
> > > > > > > > translate
> > > > > > > > the hibernate internal classes into normal java
util classes.
> > > > > > > >
> > > > > > > > [1] https://code.google.com/p/dehibernator/
> > > > > > >
> > > > > > > Well, not sure about that from performance point of
view.
> > > > > > >
> > > > > > > If we really move away from GWT completely in 4.0,
then
> > > > > > > implementing
> > > > > > > all
> > > > > > > hibernate inner collections in "uioverride"
in a similar way as
> > > > > > > Tomas
> > > > > > > did
> > > > > > > seems to me as the best solution from performance and
"easiest to
> > > > > > > remove
> > > > > > > when not needed" point of view.
> > > > > >
> > > > > > I took some inspiration from that code and wrote my own
fairly
> > > > > > simple
> > > > > > hibernate persistent collection replacer, and I put the
patch up
> > > > > > here
> > > > > > [3].
> > > > > > It
> > > > > > still uses reflection so it is probably not the fastest
thing ever
> > > > > > written.
> > > > > >
> > > > > > [3] https://gerrit.ovirt.org/#/c/41810/
> > > > > >
> > > > > > > > > > Alexander
> > > > > > > > > >
> > > > > > > > > > > Any better ideas?
> > > > > > > > > > > Thanx,
> > > > > > > > > > > Tomas
> > > > > > > > > > >
_______________________________________________
> > > > > > > > > > > Devel mailing list
> > > > > > > > > > > Devel(a)ovirt.org
> > > > > > > > > > >
http://lists.ovirt.org/mailman/listinfo/devel
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Devel mailing list
> > > > > > > > Devel(a)ovirt.org
> > > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > > >
> > > > > > _______________________________________________
> > > > > > Devel mailing list
> > > > > > Devel(a)ovirt.org
> > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > >
> > > > > _______________________________________________
> > > > > Devel mailing list
> > > > > Devel(a)ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/devel
> >
> >
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
9:28 a.m.
On Tuesday, June 02, 2015 09:10:38 AM Oved Ourfali wrote:
----- Original Message -----
> From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> To: awels(a)redhat.com
> Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, "Oved Ourfali"
> <ovedo(a)redhat.com>, devel(a)ovirt.org Sent: Tuesday, June 2, 2015 4:08:05
> PM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
>
>
> ----- Original Message -----
>
> > From: "Alexander Wels" <awels(a)redhat.com>
> > To: "Liran Zelkha" <lzelkha(a)redhat.com>
> > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Oved
Ourfali"
> > <ovedo(a)redhat.com>, devel(a)ovirt.org
> > Sent: Tuesday, June 2, 2015 2:51:31 PM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> > On Tuesday, June 02, 2015 03:05:11 AM Liran Zelkha wrote:
> > > ----- Original Message -----
> > >
> > > > From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > > To: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > Cc: awels(a)redhat.com, "Liran Zelkha"
<lzelkha(a)redhat.com>,
> > > > devel(a)ovirt.org
> > > > Sent: Tuesday, June 2, 2015 9:59:17 AM
> > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag
sent
> > > > to
> > > > FE
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > >
> > > > > From: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > > To: awels(a)redhat.com
> > > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > > Sent: Tuesday, June 2, 2015 8:35:56 AM
> > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag sent
> > > > > to
> > > > > FE
> > > > >
> > > > > top-posting:
> > > > > Due to the fact that we'll see that all over the place, I
really
> > > > > think
> > > > > that
> > > > > it would be best to support that at the frontend level, and not
> > > > > the
> > > > > backend
> > > > > level.
> > > >
> > > > well, from philosophical perspective I think that sending
> > > > org.hibernate.collection.**internal**.PersistentBag from backend to
> > > > FE
> > > > and
> > > > than faking it's implementation on FE
> > > > is a way to hell. It will bring lots of problems. For example the FE
> > > > will
> > > > be directly dependent on the internal structure of an internal
> > > > hibernate
> > > > class and if we update hibernate, FE can fail miserably.
> > > > Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems to be
> > > > more
> > > > tricky than expected - it seems to be working but Omer had issues
> > > > with
> > > > it
> > > > and in some cases (when the delegating also retainAll()) it caused
> > > > my
> > > > JRE
> > > > to fail on SIGSEGV.
> > > > It can be fixed on FE but it is a hack with all risks this kinds of
> > > > hacks
> > > > brings.
> > >
> > > I don't think it really matters where we do it, as long as server
code
> > > will
> > > still use the PersistentBag (otherwise we won't be able to use
> > > attached
> > > entities, and performance will suffer). I believe that serializing
> > > these
> > > objects to JSON/XML - will work, we won't need to do 2 level of
> > > conversions
> > > (PersistentBag-->List-->JSON/XML). So if the issue is only on the
GWT
> > > side,
> > > and we won't to write this code once, and not per entity - where is
> > > the
> > > best way to put it?
> >
> > The original problem comes from the fact that we are serializing an
> > object
> > from the backend to the frontend that the frontend cannot handle (The
> > persistent bag). This is a backend implementation detail that the
> > frontend
> > really doesn't care about. In essence GWT is Javascript that is made to
> > look
> > like Java, but it has a bunch of limitations due to the fact it still
> > ends
> > up
> > as javascript in the end.
> >
> > > > > Doing it in the backend level will cause a lot of overhead.
> > > >
> > > > Is it really lots of overhead? Comparing to all other layers the
> > > > data
> > > > has
> > > > to pass anyway? When abandoning GWT FE and start to use REST we will
> > > > anyway remove the GenericApiGWTServiceImpl which contains this
> > > > cleaning
> > > > (https://gerrit.ovirt.org/#/c/41810/).
> >
> > The overhead comes from having to scan through every single object that
> > we
> > are
> > transferring to the front end, even though right now only one or two
> > object
> > types have this particular problem. The scanning only happens if we
> > actually
> > transfer the object to the frontend. If there was a way to determine if
> > we
> > need to scan the objects being returned to the front end that would cut
> > down
> > on the overhead. Another question to ask, is the overhead BAD enough to
> > care?
> > I honestly don't know without doing some profiling. But my gut is
> > telling
> > me
> > no
> > it is not.
>
> I agree this is the correct fix on the correct place. On the other hand,
> it
> needs some optimizations and tests if it actually works and if it is not a
> big bottleneck.
> Also we need to consider if we want to have some annotation on classes we
> want to filter before sending back to frontend or we want this to be done
> all the time.
>
> Long story short, I think the patch which broke the frontend debugging
> should be reverted and merged once the
> https://gerrit.ovirt.org/#/c/41810/ is finished and tested.
What's the ETA on finishing https://gerrit.ovirt.org/#/c/41810 ?
I will be working on 41810 to make sure it is solid and is working as
expected. I am expecting it to take 1-2 days to fully complete, but I will try
to do it asap.
> @Liran: could you please revert your patch? Because currently
the FE
> debugging is pretty much broken (and I don't trust the web mode either...)
> and the solution is not completely clear now.
Let's see what the ETA is, as perhaps we can merge something soon.
Liran's patch is a big change in all DB layers, so won't be easy to revert.
> > > > > I'll leave the technical details to the UX experts here to
see
> > > > > what's
> > > > > the
> > > > > right approach to do it in the frontend side.
> > > > >
> > > > > Thanks,
> > > > > Oved
> > > > >
> > > > > ----- Original Message -----
> > > > >
> > > > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > > > To: "Martin Perina" <mperina(a)redhat.com>
> > > > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > > > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > sent
> > > > > > to
> > > > > > FE
> > > > > >
> > > > > > On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> > > > > > > ----- Original Message -----
> > > > > > >
> > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > To: "Tomas Jelinek"
<tjelinek(a)redhat.com>
> > > > > > > > Cc: "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > > > > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > > > > > Subject: Re: [ovirt-devel] hibernate's
internal
> > > > > > > > PersistentBag
> > > > > > > > sent
> > > > > > > > to
> > > > > > > > FE
> > > > > > > >
> > > > > > > > On Monday, June 01, 2015 09:33:07 AM Tomas
Jelinek wrote:
> > > > > > > > > ----- Original Message -----
> > > > > > > > >
> > > > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > > > To: devel(a)ovirt.org
> > > > > > > > > > Cc: "Tomas Jelinek"
<tjelinek(a)redhat.com>, "Liran
> > > > > > > > > > Zelkha"
> > > > > > > > > > <lzelkha(a)redhat.com> Sent:
Monday, June 1, 2015 3:16:34
> > > > > > > > > > PM
> > > > > > > > > > Subject: Re: [ovirt-devel]
hibernate's internal
> > > > > > > > > > PersistentBag
> > > > > > > > > > sent
> > > > > > > > > > to
> > > > > > > > > > FE
> > > > > > > > > >
> > > > > > > > > > On Monday, June 01, 2015 09:08:50 AM
Tomas Jelinek
wrote:
> > > > > > > > > > > Hey all,
> > > > > > > > > > >
> > > > > > > > > > > since the
org.ovirt.engine.core.common.job.Job/Step...
> > > > > > > > > > > has
> > > > > > > > > > > been
> > > > > > > > > > > moved
> > > > > > > > > > > to
> > > > > > > > > > > use
> > > > > > > > > > > the JPA we have a problem on
frontend. The problem is
> > > > > > > > > > > that
> > > > > > > > > > > the
> > > > > > > > > > > @OneToMany
> > > > > > > > > > > annotations results in a List
which is of type
> > > > > > > > > > > PersistentBag.
> > > > > > > > > > > When
> > > > > > > > > > > we
> > > > > > > > > > > send
> > > > > > > > > > > this to Frontend it fails during
deserialization. It
> > > > > > > > > > > actually
> > > > > > > > > > > fails
> > > > > > > > > > > quite
> > > > > > > > > > > bad because the FE already has an
ui-override of it
> > > > > > > > > > > which
> > > > > > > > > > > is
> > > > > > > > > > > not
> > > > > > > > > > > correct
> > > > > > > > > > > resulting in a ton of NPEs in
development mode.
> > > > > > > > > > >
> > > > > > > > > > > So, there are 2 nasty fixes I have
made where none of
> > > > > > > > > > > them
> > > > > > > > > > > should
> > > > > > > > > > > be
> > > > > > > > > > > merged
> > > > > > > > > > > but demonstrate the possibilities:
1: extend the FE to
> > > > > > > > > > > be
> > > > > > > > > > > able
> > > > > > > > > > > to
> > > > > > > > > > > work
> > > > > > > > > > > with
> > > > > > > > > > > the PersistentBag
> > > > > > > > > > >
(https://gerrit.ovirt.org/#/c/41682/)
> > > > > > > > > > > not
> > > > > > > > > > > really
> > > > > > > > > > > good
> > > > > > > > > > > solution since the PersistenBag is
an internal
> > > > > > > > > > > Hibernate
> > > > > > > > > > > class
> > > > > > > > > > > which
> > > > > > > > > > > is
> > > > > > > > > > > really not meant to be passed
around
> > > > > > > > > > >
> > > > > > > > > > > 2: fix on the backend to not send
the PersistentBag
> > > > > > > > > > > but
> > > > > > > > > > > an
> > > > > > > > > > > ArrayList.
> > > > > > > > > > > This
> > > > > > > > > > > is only a PoC fixed on a command
we face the problem
> > > > > > > > > > >
(https://gerrit.ovirt.org/#/c/41797/) Obviously this
> > > > > > > > > > > is
> > > > > > > > > > > not
> > > > > > > > > > > going
> > > > > > > > > > > to
> > > > > > > > > > > work
> > > > > > > > > > > for other commands accessing the
same Job nor for
> > > > > > > > > > > other
> > > > > > > > > > > entities.
> > > > > > > > > > >
> > > > > > > > > > > So, the first option is generic
but very very bad. The
> > > > > > > > > > > second
> > > > > > > > > > > option
> > > > > > > > > > > should
> > > > > > > > > > > be used but not sure how to do
this in a cheep way
> > > > > > > > > > > (e.g.
> > > > > > > > > > > without
> > > > > > > > > > > using
> > > > > > > > > > > reflection to deep traverse
everything sent back to
> > > > > > > > > > > frontend
> > > > > > > > > > > checking
> > > > > > > > > > > if
> > > > > > > > > > > it
> > > > > > > > > > > does not have a PersistentBag in
it.
> > > > > > > > > >
> > > > > > > > > > Tomas,
> > > > > > > > > >
> > > > > > > > > > Thanks, I was investigating the same
issue, I noticed it
> > > > > > > > > > last
> > > > > > > > > > Friday
> > > > > > > > > > just
> > > > > > > > > > before leaving, so I was investigating
the problem to
> > > > > > > > > > see
> > > > > > > > > > what
> > > > > > > > > > was
> > > > > > > > > > going
> > > > > > > > > > on. You are right we should not be
sending PersistentBag
> > > > > > > > > > to
> > > > > > > > > > the
> > > > > > > > > > frontend
> > > > > > > > > > at all. So how about we do a
combination of [1] and [2],
> > > > > > > > > > but
> > > > > > > > > > instead
> > > > > > > > > > of
> > > > > > > > > > delegating in [1] we actually simple
throw an exception
> > > > > > > > > > stating
> > > > > > > > > > don't
> > > > > > > > > > sent PersistentBag to the front end.
that way anyone
> > > > > > > > > > inadvertently
> > > > > > > > > > using
> > > > > > > > > > it will be notified immediately (since
their code won't
> > > > > > > > > > work).
> > > > > > > > >
> > > > > > > > > Throwing the exception would help us in
debugging but the
> > > > > > > > > main
> > > > > > > > > question
> > > > > > > > > is
> > > > > > > > > how will we make it work? Since we are
planning to move
> > > > > > > > > more
> > > > > > > > > and
> > > > > > > > > more
> > > > > > > > > to
> > > > > > > > > JPA so we will face this issue more and more
often.
> > > > > > > > > Solving
> > > > > > > > > it
> > > > > > > > > one
> > > > > > > > > by
> > > > > > > > > one
> > > > > > > > > on backend in each command is not going to
work.
> > > > > > > >
> > > > > > > > How about something like this [1]? It appears to
use some
> > > > > > > > aspects
> > > > > > > > to
> > > > > > > > translate
> > > > > > > > the hibernate internal classes into normal java
util
> > > > > > > > classes.
> > > > > > > >
> > > > > > > > [1] https://code.google.com/p/dehibernator/
> > > > > > >
> > > > > > > Well, not sure about that from performance point of
view.
> > > > > > >
> > > > > > > If we really move away from GWT completely in 4.0,
then
> > > > > > > implementing
> > > > > > > all
> > > > > > > hibernate inner collections in "uioverride"
in a similar way
> > > > > > > as
> > > > > > > Tomas
> > > > > > > did
> > > > > > > seems to me as the best solution from performance and
"easiest
> > > > > > > to
> > > > > > > remove
> > > > > > > when not needed" point of view.
> > > > > >
> > > > > > I took some inspiration from that code and wrote my own
fairly
> > > > > > simple
> > > > > > hibernate persistent collection replacer, and I put the
patch up
> > > > > > here
> > > > > > [3].
> > > > > > It
> > > > > > still uses reflection so it is probably not the fastest
thing
> > > > > > ever
> > > > > > written.
> > > > > >
> > > > > > [3] https://gerrit.ovirt.org/#/c/41810/
> > > > > >
> > > > > > > > > > Alexander
> > > > > > > > > >
> > > > > > > > > > > Any better ideas?
> > > > > > > > > > > Thanx,
> > > > > > > > > > > Tomas
> > > > > > > > > > >
_______________________________________________
> > > > > > > > > > > Devel mailing list
> > > > > > > > > > > Devel(a)ovirt.org
> > > > > > > > > > >
http://lists.ovirt.org/mailman/listinfo/devel
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Devel mailing list
> > > > > > > > Devel(a)ovirt.org
> > > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > > >
> > > > > > _______________________________________________
> > > > > > Devel mailing list
> > > > > > Devel(a)ovirt.org
> > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > >
> > > > > _______________________________________________
> > > > > Devel mailing list
> > > > > Devel(a)ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/devel
2:58 a.m.
----- Original Message -----
From: "Alexander Wels" <awels(a)redhat.com>
To: "Oved Ourfali" <ovedo(a)redhat.com>
Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
Sent: Tuesday, June 2, 2015 3:28:35 PM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
On Tuesday, June 02, 2015 09:10:38 AM Oved Ourfali wrote:
> ----- Original Message -----
>
> > From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > To: awels(a)redhat.com
> > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, "Oved
Ourfali"
> > <ovedo(a)redhat.com>, devel(a)ovirt.org Sent: Tuesday, June 2, 2015 4:08:05
> > PM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> >
> >
> > ----- Original Message -----
> >
> > > From: "Alexander Wels" <awels(a)redhat.com>
> > > To: "Liran Zelkha" <lzelkha(a)redhat.com>
> > > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Oved
Ourfali"
> > > <ovedo(a)redhat.com>, devel(a)ovirt.org
> > > Sent: Tuesday, June 2, 2015 2:51:31 PM
> > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to
> > > FE
> > >
> > > On Tuesday, June 02, 2015 03:05:11 AM Liran Zelkha wrote:
> > > > ----- Original Message -----
> > > >
> > > > > From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > > > To: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > > Cc: awels(a)redhat.com, "Liran Zelkha"
<lzelkha(a)redhat.com>,
> > > > > devel(a)ovirt.org
> > > > > Sent: Tuesday, June 2, 2015 9:59:17 AM
> > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag sent
> > > > > to
> > > > > FE
> > > > >
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > >
> > > > > > From: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > > > To: awels(a)redhat.com
> > > > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > > > Sent: Tuesday, June 2, 2015 8:35:56 AM
> > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > sent
> > > > > > to
> > > > > > FE
> > > > > >
> > > > > > top-posting:
> > > > > > Due to the fact that we'll see that all over the place,
I really
> > > > > > think
> > > > > > that
> > > > > > it would be best to support that at the frontend level, and
not
> > > > > > the
> > > > > > backend
> > > > > > level.
> > > > >
> > > > > well, from philosophical perspective I think that sending
> > > > > org.hibernate.collection.**internal**.PersistentBag from backend
to
> > > > > FE
> > > > > and
> > > > > than faking it's implementation on FE
> > > > > is a way to hell. It will bring lots of problems. For example
the
> > > > > FE
> > > > > will
> > > > > be directly dependent on the internal structure of an internal
> > > > > hibernate
> > > > > class and if we update hibernate, FE can fail miserably.
> > > > > Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems
to
> > > > > be
> > > > > more
> > > > > tricky than expected - it seems to be working but Omer had
issues
> > > > > with
> > > > > it
> > > > > and in some cases (when the delegating also retainAll()) it
caused
> > > > > my
> > > > > JRE
> > > > > to fail on SIGSEGV.
> > > > > It can be fixed on FE but it is a hack with all risks this kinds
of
> > > > > hacks
> > > > > brings.
> > > >
> > > > I don't think it really matters where we do it, as long as
server
> > > > code
> > > > will
> > > > still use the PersistentBag (otherwise we won't be able to use
> > > > attached
> > > > entities, and performance will suffer). I believe that serializing
> > > > these
> > > > objects to JSON/XML - will work, we won't need to do 2 level of
> > > > conversions
> > > > (PersistentBag-->List-->JSON/XML). So if the issue is only on
the GWT
> > > > side,
> > > > and we won't to write this code once, and not per entity - where
is
> > > > the
> > > > best way to put it?
> > >
> > > The original problem comes from the fact that we are serializing an
> > > object
> > > from the backend to the frontend that the frontend cannot handle (The
> > > persistent bag). This is a backend implementation detail that the
> > > frontend
> > > really doesn't care about. In essence GWT is Javascript that is made
> > > to
> > > look
> > > like Java, but it has a bunch of limitations due to the fact it still
> > > ends
> > > up
> > > as javascript in the end.
> > >
> > > > > > Doing it in the backend level will cause a lot of
overhead.
> > > > >
> > > > > Is it really lots of overhead? Comparing to all other layers
the
> > > > > data
> > > > > has
> > > > > to pass anyway? When abandoning GWT FE and start to use REST we
> > > > > will
> > > > > anyway remove the GenericApiGWTServiceImpl which contains this
> > > > > cleaning
> > > > > (https://gerrit.ovirt.org/#/c/41810/).
> > >
> > > The overhead comes from having to scan through every single object that
> > > we
> > > are
> > > transferring to the front end, even though right now only one or two
> > > object
> > > types have this particular problem. The scanning only happens if we
> > > actually
> > > transfer the object to the frontend. If there was a way to determine if
> > > we
> > > need to scan the objects being returned to the front end that would cut
> > > down
> > > on the overhead. Another question to ask, is the overhead BAD enough to
> > > care?
> > > I honestly don't know without doing some profiling. But my gut is
> > > telling
> > > me
> > > no
> > > it is not.
> >
> > I agree this is the correct fix on the correct place. On the other hand,
> > it
> > needs some optimizations and tests if it actually works and if it is not
> > a
> > big bottleneck.
> > Also we need to consider if we want to have some annotation on classes we
> > want to filter before sending back to frontend or we want this to be done
> > all the time.
> >
> > Long story short, I think the patch which broke the frontend debugging
> > should be reverted and merged once the
> > https://gerrit.ovirt.org/#/c/41810/ is finished and tested.
>
> What's the ETA on finishing https://gerrit.ovirt.org/#/c/41810 ?
>
I will be working on 41810 to make sure it is solid and is working as
expected. I am expecting it to take 1-2 days to fully complete, but I will
try
to do it asap.
OK, Alexander have posted a new version of https://gerrit.ovirt.org/#/c/41810 and I think
it is now good enough to be merged. I have tested it on my setup and everything seems to
be working again.
I think there is some room for enhancements but that enhancements can be discussed
separately in a separate patch.
Long story short - everyone who was stuck can now debug the FE again :) Thanx Alexander
for the patch!
> > @Liran: could you please revert your patch? Because currently the FE
> > debugging is pretty much broken (and I don't trust the web mode
> > either...)
> > and the solution is not completely clear now.
>
> Let's see what the ETA is, as perhaps we can merge something soon.
> Liran's patch is a big change in all DB layers, so won't be easy to revert.
>
> > > > > > I'll leave the technical details to the UX experts here
to see
> > > > > > what's
> > > > > > the
> > > > > > right approach to do it in the frontend side.
> > > > > >
> > > > > > Thanks,
> > > > > > Oved
> > > > > >
> > > > > > ----- Original Message -----
> > > > > >
> > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > To: "Martin Perina"
<mperina(a)redhat.com>
> > > > > > > Cc: "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > > > > > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > > sent
> > > > > > > to
> > > > > > > FE
> > > > > > >
> > > > > > > On Monday, June 01, 2015 10:51:28 AM Martin Perina
wrote:
> > > > > > > > ----- Original Message -----
> > > > > > > >
> > > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > > To: "Tomas Jelinek"
<tjelinek(a)redhat.com>
> > > > > > > > > Cc: "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > > > > > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > > > > > > Subject: Re: [ovirt-devel] hibernate's
internal
> > > > > > > > > PersistentBag
> > > > > > > > > sent
> > > > > > > > > to
> > > > > > > > > FE
> > > > > > > > >
> > > > > > > > > On Monday, June 01, 2015 09:33:07 AM Tomas
Jelinek wrote:
> > > > > > > > > > ----- Original Message -----
> > > > > > > > > >
> > > > > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > > > > To: devel(a)ovirt.org
> > > > > > > > > > > Cc: "Tomas Jelinek"
<tjelinek(a)redhat.com>, "Liran
> > > > > > > > > > > Zelkha"
> > > > > > > > > > > <lzelkha(a)redhat.com> Sent:
Monday, June 1, 2015 3:16:34
> > > > > > > > > > > PM
> > > > > > > > > > > Subject: Re: [ovirt-devel]
hibernate's internal
> > > > > > > > > > > PersistentBag
> > > > > > > > > > > sent
> > > > > > > > > > > to
> > > > > > > > > > > FE
> > > > > > > > > > >
> > > > > > > > > > > On Monday, June 01, 2015 09:08:50
AM Tomas Jelinek
wrote:
> > > > > > > > > > > > Hey all,
> > > > > > > > > > > >
> > > > > > > > > > > > since the
> > > > > > > > > > > >
org.ovirt.engine.core.common.job.Job/Step...
> > > > > > > > > > > > has
> > > > > > > > > > > > been
> > > > > > > > > > > > moved
> > > > > > > > > > > > to
> > > > > > > > > > > > use
> > > > > > > > > > > > the JPA we have a problem on
frontend. The problem is
> > > > > > > > > > > > that
> > > > > > > > > > > > the
> > > > > > > > > > > > @OneToMany
> > > > > > > > > > > > annotations results in a List
which is of type
> > > > > > > > > > > > PersistentBag.
> > > > > > > > > > > > When
> > > > > > > > > > > > we
> > > > > > > > > > > > send
> > > > > > > > > > > > this to Frontend it fails
during deserialization. It
> > > > > > > > > > > > actually
> > > > > > > > > > > > fails
> > > > > > > > > > > > quite
> > > > > > > > > > > > bad because the FE already
has an ui-override of it
> > > > > > > > > > > > which
> > > > > > > > > > > > is
> > > > > > > > > > > > not
> > > > > > > > > > > > correct
> > > > > > > > > > > > resulting in a ton of NPEs in
development mode.
> > > > > > > > > > > >
> > > > > > > > > > > > So, there are 2 nasty fixes I
have made where none of
> > > > > > > > > > > > them
> > > > > > > > > > > > should
> > > > > > > > > > > > be
> > > > > > > > > > > > merged
> > > > > > > > > > > > but demonstrate the
possibilities: 1: extend the FE
> > > > > > > > > > > > to
> > > > > > > > > > > > be
> > > > > > > > > > > > able
> > > > > > > > > > > > to
> > > > > > > > > > > > work
> > > > > > > > > > > > with
> > > > > > > > > > > > the PersistentBag
> > > > > > > > > > > >
(https://gerrit.ovirt.org/#/c/41682/)
> > > > > > > > > > > > not
> > > > > > > > > > > > really
> > > > > > > > > > > > good
> > > > > > > > > > > > solution since the
PersistenBag is an internal
> > > > > > > > > > > > Hibernate
> > > > > > > > > > > > class
> > > > > > > > > > > > which
> > > > > > > > > > > > is
> > > > > > > > > > > > really not meant to be passed
around
> > > > > > > > > > > >
> > > > > > > > > > > > 2: fix on the backend to not
send the PersistentBag
> > > > > > > > > > > > but
> > > > > > > > > > > > an
> > > > > > > > > > > > ArrayList.
> > > > > > > > > > > > This
> > > > > > > > > > > > is only a PoC fixed on a
command we face the problem
> > > > > > > > > > > >
(https://gerrit.ovirt.org/#/c/41797/) Obviously this
> > > > > > > > > > > > is
> > > > > > > > > > > > not
> > > > > > > > > > > > going
> > > > > > > > > > > > to
> > > > > > > > > > > > work
> > > > > > > > > > > > for other commands accessing
the same Job nor for
> > > > > > > > > > > > other
> > > > > > > > > > > > entities.
> > > > > > > > > > > >
> > > > > > > > > > > > So, the first option is
generic but very very bad.
> > > > > > > > > > > > The
> > > > > > > > > > > > second
> > > > > > > > > > > > option
> > > > > > > > > > > > should
> > > > > > > > > > > > be used but not sure how to
do this in a cheep way
> > > > > > > > > > > > (e.g.
> > > > > > > > > > > > without
> > > > > > > > > > > > using
> > > > > > > > > > > > reflection to deep traverse
everything sent back to
> > > > > > > > > > > > frontend
> > > > > > > > > > > > checking
> > > > > > > > > > > > if
> > > > > > > > > > > > it
> > > > > > > > > > > > does not have a PersistentBag
in it.
> > > > > > > > > > >
> > > > > > > > > > > Tomas,
> > > > > > > > > > >
> > > > > > > > > > > Thanks, I was investigating the
same issue, I noticed
> > > > > > > > > > > it
> > > > > > > > > > > last
> > > > > > > > > > > Friday
> > > > > > > > > > > just
> > > > > > > > > > > before leaving, so I was
investigating the problem to
> > > > > > > > > > > see
> > > > > > > > > > > what
> > > > > > > > > > > was
> > > > > > > > > > > going
> > > > > > > > > > > on. You are right we should not be
sending
> > > > > > > > > > > PersistentBag
> > > > > > > > > > > to
> > > > > > > > > > > the
> > > > > > > > > > > frontend
> > > > > > > > > > > at all. So how about we do a
combination of [1] and
> > > > > > > > > > > [2],
> > > > > > > > > > > but
> > > > > > > > > > > instead
> > > > > > > > > > > of
> > > > > > > > > > > delegating in [1] we actually
simple throw an exception
> > > > > > > > > > > stating
> > > > > > > > > > > don't
> > > > > > > > > > > sent PersistentBag to the front
end. that way anyone
> > > > > > > > > > > inadvertently
> > > > > > > > > > > using
> > > > > > > > > > > it will be notified immediately
(since their code won't
> > > > > > > > > > > work).
> > > > > > > > > >
> > > > > > > > > > Throwing the exception would help us in
debugging but the
> > > > > > > > > > main
> > > > > > > > > > question
> > > > > > > > > > is
> > > > > > > > > > how will we make it work? Since we are
planning to move
> > > > > > > > > > more
> > > > > > > > > > and
> > > > > > > > > > more
> > > > > > > > > > to
> > > > > > > > > > JPA so we will face this issue more and
more often.
> > > > > > > > > > Solving
> > > > > > > > > > it
> > > > > > > > > > one
> > > > > > > > > > by
> > > > > > > > > > one
> > > > > > > > > > on backend in each command is not going
to work.
> > > > > > > > >
> > > > > > > > > How about something like this [1]? It
appears to use some
> > > > > > > > > aspects
> > > > > > > > > to
> > > > > > > > > translate
> > > > > > > > > the hibernate internal classes into normal
java util
> > > > > > > > > classes.
> > > > > > > > >
> > > > > > > > > [1] https://code.google.com/p/dehibernator/
> > > > > > > >
> > > > > > > > Well, not sure about that from performance point
of view.
> > > > > > > >
> > > > > > > > If we really move away from GWT completely in
4.0, then
> > > > > > > > implementing
> > > > > > > > all
> > > > > > > > hibernate inner collections in
"uioverride" in a similar way
> > > > > > > > as
> > > > > > > > Tomas
> > > > > > > > did
> > > > > > > > seems to me as the best solution from performance
and
> > > > > > > > "easiest
> > > > > > > > to
> > > > > > > > remove
> > > > > > > > when not needed" point of view.
> > > > > > >
> > > > > > > I took some inspiration from that code and wrote my
own fairly
> > > > > > > simple
> > > > > > > hibernate persistent collection replacer, and I put
the patch
> > > > > > > up
> > > > > > > here
> > > > > > > [3].
> > > > > > > It
> > > > > > > still uses reflection so it is probably not the
fastest thing
> > > > > > > ever
> > > > > > > written.
> > > > > > >
> > > > > > > [3] https://gerrit.ovirt.org/#/c/41810/
> > > > > > >
> > > > > > > > > > > Alexander
> > > > > > > > > > >
> > > > > > > > > > > > Any better ideas?
> > > > > > > > > > > > Thanx,
> > > > > > > > > > > > Tomas
> > > > > > > > > > > >
_______________________________________________
> > > > > > > > > > > > Devel mailing list
> > > > > > > > > > > > Devel(a)ovirt.org
> > > > > > > > > > > >
http://lists.ovirt.org/mailman/listinfo/devel
> > > > > > > > >
> > > > > > > > >
_______________________________________________
> > > > > > > > > Devel mailing list
> > > > > > > > > Devel(a)ovirt.org
> > > > > > > > >
http://lists.ovirt.org/mailman/listinfo/devel
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > Devel mailing list
> > > > > > > Devel(a)ovirt.org
> > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > > >
> > > > > > _______________________________________________
> > > > > > Devel mailing list
> > > > > > Devel(a)ovirt.org
> > > > > > http://lists.ovirt.org/mailman/listinfo/devel
3:06 a.m.
----- Original Message -----
From: "Tomas Jelinek" <tjelinek(a)redhat.com>
To: awels(a)redhat.com
Cc: "Oved Ourfali" <ovedo(a)redhat.com>, "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
Sent: Wednesday, June 3, 2015 9:58:34 AM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
----- Original Message -----
> From: "Alexander Wels" <awels(a)redhat.com>
> To: "Oved Ourfali" <ovedo(a)redhat.com>
> Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Liran Zelkha"
> <lzelkha(a)redhat.com>, devel(a)ovirt.org
> Sent: Tuesday, June 2, 2015 3:28:35 PM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
> On Tuesday, June 02, 2015 09:10:38 AM Oved Ourfali wrote:
> > ----- Original Message -----
> >
> > > From: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > To: awels(a)redhat.com
> > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, "Oved
Ourfali"
> > > <ovedo(a)redhat.com>, devel(a)ovirt.org Sent: Tuesday, June 2, 2015
4:08:05
> > > PM
> > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to
> > > FE
> > >
> > >
> > >
> > > ----- Original Message -----
> > >
> > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > To: "Liran Zelkha" <lzelkha(a)redhat.com>
> > > > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>, "Oved
Ourfali"
> > > > <ovedo(a)redhat.com>, devel(a)ovirt.org
> > > > Sent: Tuesday, June 2, 2015 2:51:31 PM
> > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag
sent to
> > > > FE
> > > >
> > > > On Tuesday, June 02, 2015 03:05:11 AM Liran Zelkha wrote:
> > > > > ----- Original Message -----
> > > > >
> > > > > > From: "Tomas Jelinek"
<tjelinek(a)redhat.com>
> > > > > > To: "Oved Ourfali" <ovedo(a)redhat.com>
> > > > > > Cc: awels(a)redhat.com, "Liran Zelkha"
<lzelkha(a)redhat.com>,
> > > > > > devel(a)ovirt.org
> > > > > > Sent: Tuesday, June 2, 2015 9:59:17 AM
> > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > sent
> > > > > > to
> > > > > > FE
> > > > > >
> > > > > >
> > > > > >
> > > > > > ----- Original Message -----
> > > > > >
> > > > > > > From: "Oved Ourfali"
<ovedo(a)redhat.com>
> > > > > > > To: awels(a)redhat.com
> > > > > > > Cc: "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > > > > > Sent: Tuesday, June 2, 2015 8:35:56 AM
> > > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > > sent
> > > > > > > to
> > > > > > > FE
> > > > > > >
> > > > > > > top-posting:
> > > > > > > Due to the fact that we'll see that all over the
place, I
> > > > > > > really
> > > > > > > think
> > > > > > > that
> > > > > > > it would be best to support that at the frontend
level, and not
> > > > > > > the
> > > > > > > backend
> > > > > > > level.
> > > > > >
> > > > > > well, from philosophical perspective I think that sending
> > > > > > org.hibernate.collection.**internal**.PersistentBag from
backend
> > > > > > to
> > > > > > FE
> > > > > > and
> > > > > > than faking it's implementation on FE
> > > > > > is a way to hell. It will bring lots of problems. For
example the
> > > > > > FE
> > > > > > will
> > > > > > be directly dependent on the internal structure of an
internal
> > > > > > hibernate
> > > > > > class and if we update hibernate, FE can fail miserably.
> > > > > > Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1)
seems to
> > > > > > be
> > > > > > more
> > > > > > tricky than expected - it seems to be working but Omer had
issues
> > > > > > with
> > > > > > it
> > > > > > and in some cases (when the delegating also retainAll())
it
> > > > > > caused
> > > > > > my
> > > > > > JRE
> > > > > > to fail on SIGSEGV.
> > > > > > It can be fixed on FE but it is a hack with all risks this
kinds
> > > > > > of
> > > > > > hacks
> > > > > > brings.
> > > > >
> > > > > I don't think it really matters where we do it, as long as
server
> > > > > code
> > > > > will
> > > > > still use the PersistentBag (otherwise we won't be able to
use
> > > > > attached
> > > > > entities, and performance will suffer). I believe that
serializing
> > > > > these
> > > > > objects to JSON/XML - will work, we won't need to do 2 level
of
> > > > > conversions
> > > > > (PersistentBag-->List-->JSON/XML). So if the issue is only
on the
> > > > > GWT
> > > > > side,
> > > > > and we won't to write this code once, and not per entity -
where is
> > > > > the
> > > > > best way to put it?
> > > >
> > > > The original problem comes from the fact that we are serializing an
> > > > object
> > > > from the backend to the frontend that the frontend cannot handle
(The
> > > > persistent bag). This is a backend implementation detail that the
> > > > frontend
> > > > really doesn't care about. In essence GWT is Javascript that is
made
> > > > to
> > > > look
> > > > like Java, but it has a bunch of limitations due to the fact it
still
> > > > ends
> > > > up
> > > > as javascript in the end.
> > > >
> > > > > > > Doing it in the backend level will cause a lot of
overhead.
> > > > > >
> > > > > > Is it really lots of overhead? Comparing to all other
layers the
> > > > > > data
> > > > > > has
> > > > > > to pass anyway? When abandoning GWT FE and start to use
REST we
> > > > > > will
> > > > > > anyway remove the GenericApiGWTServiceImpl which contains
this
> > > > > > cleaning
> > > > > > (https://gerrit.ovirt.org/#/c/41810/).
> > > >
> > > > The overhead comes from having to scan through every single object
> > > > that
> > > > we
> > > > are
> > > > transferring to the front end, even though right now only one or two
> > > > object
> > > > types have this particular problem. The scanning only happens if we
> > > > actually
> > > > transfer the object to the frontend. If there was a way to determine
> > > > if
> > > > we
> > > > need to scan the objects being returned to the front end that would
> > > > cut
> > > > down
> > > > on the overhead. Another question to ask, is the overhead BAD enough
> > > > to
> > > > care?
> > > > I honestly don't know without doing some profiling. But my gut
is
> > > > telling
> > > > me
> > > > no
> > > > it is not.
> > >
> > > I agree this is the correct fix on the correct place. On the other
> > > hand,
> > > it
> > > needs some optimizations and tests if it actually works and if it is
> > > not
> > > a
> > > big bottleneck.
> > > Also we need to consider if we want to have some annotation on classes
> > > we
> > > want to filter before sending back to frontend or we want this to be
> > > done
> > > all the time.
> > >
> > > Long story short, I think the patch which broke the frontend debugging
> > > should be reverted and merged once the
> > > https://gerrit.ovirt.org/#/c/41810/ is finished and tested.
> >
> > What's the ETA on finishing https://gerrit.ovirt.org/#/c/41810 ?
> >
>
> I will be working on 41810 to make sure it is solid and is working as
> expected. I am expecting it to take 1-2 days to fully complete, but I will
> try
> to do it asap.
OK, Alexander have posted a new version of https://gerrit.ovirt.org/#/c/41810
and I think it is now good enough to be merged. I have tested it on my setup
and everything seems to be working again.
I think there is some room for enhancements but that enhancements can be
discussed separately in a separate patch.
Long story short - everyone who was stuck can now debug the FE again :) Thanx
Alexander for the patch!
Thank you both for the quick analysis and the solution!
>
> > > @Liran: could you please revert your patch? Because currently the FE
> > > debugging is pretty much broken (and I don't trust the web mode
> > > either...)
> > > and the solution is not completely clear now.
> >
> > Let's see what the ETA is, as perhaps we can merge something soon.
> > Liran's patch is a big change in all DB layers, so won't be easy to
> > revert.
> >
> > > > > > > I'll leave the technical details to the UX experts
here to see
> > > > > > > what's
> > > > > > > the
> > > > > > > right approach to do it in the frontend side.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Oved
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > >
> > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > To: "Martin Perina"
<mperina(a)redhat.com>
> > > > > > > > Cc: "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > > > > > > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > > > > > > > Subject: Re: [ovirt-devel] hibernate's
internal PersistentBag
> > > > > > > > sent
> > > > > > > > to
> > > > > > > > FE
> > > > > > > >
> > > > > > > > On Monday, June 01, 2015 10:51:28 AM Martin
Perina wrote:
> > > > > > > > > ----- Original Message -----
> > > > > > > > >
> > > > > > > > > > From: "Alexander Wels"
<awels(a)redhat.com>
> > > > > > > > > > To: "Tomas Jelinek"
<tjelinek(a)redhat.com>
> > > > > > > > > > Cc: "Liran Zelkha"
<lzelkha(a)redhat.com>, devel(a)ovirt.org
> > > > > > > > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > > > > > > > Subject: Re: [ovirt-devel]
hibernate's internal
> > > > > > > > > > PersistentBag
> > > > > > > > > > sent
> > > > > > > > > > to
> > > > > > > > > > FE
> > > > > > > > > >
> > > > > > > > > > On Monday, June 01, 2015 09:33:07 AM
Tomas Jelinek wrote:
> > > > > > > > > > > ----- Original Message -----
> > > > > > > > > > >
> > > > > > > > > > > > From: "Alexander
Wels" <awels(a)redhat.com>
> > > > > > > > > > > > To: devel(a)ovirt.org
> > > > > > > > > > > > Cc: "Tomas Jelinek"
<tjelinek(a)redhat.com>, "Liran
> > > > > > > > > > > > Zelkha"
> > > > > > > > > > > > <lzelkha(a)redhat.com>
Sent: Monday, June 1, 2015
> > > > > > > > > > > > 3:16:34
> > > > > > > > > > > > PM
> > > > > > > > > > > > Subject: Re: [ovirt-devel]
hibernate's internal
> > > > > > > > > > > > PersistentBag
> > > > > > > > > > > > sent
> > > > > > > > > > > > to
> > > > > > > > > > > > FE
> > > > > > > > > > > >
> > > > > > > > > > > > On Monday, June 01, 2015
09:08:50 AM Tomas Jelinek
> wrote:
> > > > > > > > > > > > > Hey all,
> > > > > > > > > > > > >
> > > > > > > > > > > > > since the
> > > > > > > > > > > > >
org.ovirt.engine.core.common.job.Job/Step...
> > > > > > > > > > > > > has
> > > > > > > > > > > > > been
> > > > > > > > > > > > > moved
> > > > > > > > > > > > > to
> > > > > > > > > > > > > use
> > > > > > > > > > > > > the JPA we have a
problem on frontend. The problem
> > > > > > > > > > > > > is
> > > > > > > > > > > > > that
> > > > > > > > > > > > > the
> > > > > > > > > > > > > @OneToMany
> > > > > > > > > > > > > annotations results in a
List which is of type
> > > > > > > > > > > > > PersistentBag.
> > > > > > > > > > > > > When
> > > > > > > > > > > > > we
> > > > > > > > > > > > > send
> > > > > > > > > > > > > this to Frontend it
fails during deserialization.
> > > > > > > > > > > > > It
> > > > > > > > > > > > > actually
> > > > > > > > > > > > > fails
> > > > > > > > > > > > > quite
> > > > > > > > > > > > > bad because the FE
already has an ui-override of it
> > > > > > > > > > > > > which
> > > > > > > > > > > > > is
> > > > > > > > > > > > > not
> > > > > > > > > > > > > correct
> > > > > > > > > > > > > resulting in a ton of
NPEs in development mode.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So, there are 2 nasty
fixes I have made where none
> > > > > > > > > > > > > of
> > > > > > > > > > > > > them
> > > > > > > > > > > > > should
> > > > > > > > > > > > > be
> > > > > > > > > > > > > merged
> > > > > > > > > > > > > but demonstrate the
possibilities: 1: extend the FE
> > > > > > > > > > > > > to
> > > > > > > > > > > > > be
> > > > > > > > > > > > > able
> > > > > > > > > > > > > to
> > > > > > > > > > > > > work
> > > > > > > > > > > > > with
> > > > > > > > > > > > > the PersistentBag
> > > > > > > > > > > > >
(https://gerrit.ovirt.org/#/c/41682/)
> > > > > > > > > > > > > not
> > > > > > > > > > > > > really
> > > > > > > > > > > > > good
> > > > > > > > > > > > > solution since the
PersistenBag is an internal
> > > > > > > > > > > > > Hibernate
> > > > > > > > > > > > > class
> > > > > > > > > > > > > which
> > > > > > > > > > > > > is
> > > > > > > > > > > > > really not meant to be
passed around
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2: fix on the backend to
not send the PersistentBag
> > > > > > > > > > > > > but
> > > > > > > > > > > > > an
> > > > > > > > > > > > > ArrayList.
> > > > > > > > > > > > > This
> > > > > > > > > > > > > is only a PoC fixed on a
command we face the
> > > > > > > > > > > > > problem
> > > > > > > > > > > > >
(https://gerrit.ovirt.org/#/c/41797/) Obviously
> > > > > > > > > > > > > this
> > > > > > > > > > > > > is
> > > > > > > > > > > > > not
> > > > > > > > > > > > > going
> > > > > > > > > > > > > to
> > > > > > > > > > > > > work
> > > > > > > > > > > > > for other commands
accessing the same Job nor for
> > > > > > > > > > > > > other
> > > > > > > > > > > > > entities.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So, the first option is
generic but very very bad.
> > > > > > > > > > > > > The
> > > > > > > > > > > > > second
> > > > > > > > > > > > > option
> > > > > > > > > > > > > should
> > > > > > > > > > > > > be used but not sure how
to do this in a cheep way
> > > > > > > > > > > > > (e.g.
> > > > > > > > > > > > > without
> > > > > > > > > > > > > using
> > > > > > > > > > > > > reflection to deep
traverse everything sent back to
> > > > > > > > > > > > > frontend
> > > > > > > > > > > > > checking
> > > > > > > > > > > > > if
> > > > > > > > > > > > > it
> > > > > > > > > > > > > does not have a
PersistentBag in it.
> > > > > > > > > > > >
> > > > > > > > > > > > Tomas,
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks, I was investigating
the same issue, I noticed
> > > > > > > > > > > > it
> > > > > > > > > > > > last
> > > > > > > > > > > > Friday
> > > > > > > > > > > > just
> > > > > > > > > > > > before leaving, so I was
investigating the problem to
> > > > > > > > > > > > see
> > > > > > > > > > > > what
> > > > > > > > > > > > was
> > > > > > > > > > > > going
> > > > > > > > > > > > on. You are right we should
not be sending
> > > > > > > > > > > > PersistentBag
> > > > > > > > > > > > to
> > > > > > > > > > > > the
> > > > > > > > > > > > frontend
> > > > > > > > > > > > at all. So how about we do a
combination of [1] and
> > > > > > > > > > > > [2],
> > > > > > > > > > > > but
> > > > > > > > > > > > instead
> > > > > > > > > > > > of
> > > > > > > > > > > > delegating in [1] we actually
simple throw an
> > > > > > > > > > > > exception
> > > > > > > > > > > > stating
> > > > > > > > > > > > don't
> > > > > > > > > > > > sent PersistentBag to the
front end. that way anyone
> > > > > > > > > > > > inadvertently
> > > > > > > > > > > > using
> > > > > > > > > > > > it will be notified
immediately (since their code
> > > > > > > > > > > > won't
> > > > > > > > > > > > work).
> > > > > > > > > > >
> > > > > > > > > > > Throwing the exception would help
us in debugging but
> > > > > > > > > > > the
> > > > > > > > > > > main
> > > > > > > > > > > question
> > > > > > > > > > > is
> > > > > > > > > > > how will we make it work? Since we
are planning to move
> > > > > > > > > > > more
> > > > > > > > > > > and
> > > > > > > > > > > more
> > > > > > > > > > > to
> > > > > > > > > > > JPA so we will face this issue
more and more often.
> > > > > > > > > > > Solving
> > > > > > > > > > > it
> > > > > > > > > > > one
> > > > > > > > > > > by
> > > > > > > > > > > one
> > > > > > > > > > > on backend in each command is not
going to work.
> > > > > > > > > >
> > > > > > > > > > How about something like this [1]? It
appears to use some
> > > > > > > > > > aspects
> > > > > > > > > > to
> > > > > > > > > > translate
> > > > > > > > > > the hibernate internal classes into
normal java util
> > > > > > > > > > classes.
> > > > > > > > > >
> > > > > > > > > > [1]
https://code.google.com/p/dehibernator/
> > > > > > > > >
> > > > > > > > > Well, not sure about that from performance
point of view.
> > > > > > > > >
> > > > > > > > > If we really move away from GWT completely
in 4.0, then
> > > > > > > > > implementing
> > > > > > > > > all
> > > > > > > > > hibernate inner collections in
"uioverride" in a similar
> > > > > > > > > way
> > > > > > > > > as
> > > > > > > > > Tomas
> > > > > > > > > did
> > > > > > > > > seems to me as the best solution from
performance and
> > > > > > > > > "easiest
> > > > > > > > > to
> > > > > > > > > remove
> > > > > > > > > when not needed" point of view.
> > > > > > > >
> > > > > > > > I took some inspiration from that code and wrote
my own
> > > > > > > > fairly
> > > > > > > > simple
> > > > > > > > hibernate persistent collection replacer, and I
put the patch
> > > > > > > > up
> > > > > > > > here
> > > > > > > > [3].
> > > > > > > > It
> > > > > > > > still uses reflection so it is probably not the
fastest thing
> > > > > > > > ever
> > > > > > > > written.
> > > > > > > >
> > > > > > > > [3] https://gerrit.ovirt.org/#/c/41810/
> > > > > > > >
> > > > > > > > > > > > Alexander
> > > > > > > > > > > >
> > > > > > > > > > > > > Any better ideas?
> > > > > > > > > > > > > Thanx,
> > > > > > > > > > > > > Tomas
> > > > > > > > > > > > >
_______________________________________________
> > > > > > > > > > > > > Devel mailing list
> > > > > > > > > > > > > Devel(a)ovirt.org
> > > > > > > > > > > > >
http://lists.ovirt.org/mailman/listinfo/devel
> > > > > > > > > >
> > > > > > > > > >
_______________________________________________
> > > > > > > > > > Devel mailing list
> > > > > > > > > > Devel(a)ovirt.org
> > > > > > > > > >
http://lists.ovirt.org/mailman/listinfo/devel
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Devel mailing list
> > > > > > > > Devel(a)ovirt.org
> > > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > Devel mailing list
> > > > > > > Devel(a)ovirt.org
> > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
>
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
3:05 a.m.
----- Original Message -----
From: "Tomas Jelinek" <tjelinek(a)redhat.com>
To: "Oved Ourfali" <ovedo(a)redhat.com>
Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
Sent: Tuesday, June 2, 2015 9:59:17 AM
Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
----- Original Message -----
> From: "Oved Ourfali" <ovedo(a)redhat.com>
> To: awels(a)redhat.com
> Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> Sent: Tuesday, June 2, 2015 8:35:56 AM
> Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
>
> top-posting:
> Due to the fact that we'll see that all over the place, I really think that
> it would be best to support that at the frontend level, and not the backend
> level.
well, from philosophical perspective I think that sending
org.hibernate.collection.**internal**.PersistentBag from backend to FE and
than faking it's implementation on FE
is a way to hell. It will bring lots of problems. For example the FE will be
directly dependent on the internal structure of an internal hibernate class
and if we update hibernate, FE can fail miserably.
Also the FE patch (https://gerrit.ovirt.org/#/c/41682/1) seems to be more
tricky than expected - it seems to be working but Omer had issues with it
and in some cases (when the delegating also retainAll()) it caused my JRE to
fail on SIGSEGV.
It can be fixed on FE but it is a hack with all risks this kinds of hacks
brings.
> Doing it in the backend level will cause a lot of overhead.
Is it really lots of overhead? Comparing to all other layers the data has to
pass anyway? When abandoning GWT FE and start to use REST we will anyway
remove the GenericApiGWTServiceImpl which contains this cleaning
(https://gerrit.ovirt.org/#/c/41810/).
I'm okay with doing this cleaning. Just not to propagate the fix to the business
entities themselves (that's what I meant by saying in the backend level).
> I'll leave the technical details to the UX experts here to see what's the
> right approach to do it in the frontend side.
>
> Thanks,
> Oved
>
> ----- Original Message -----
> > From: "Alexander Wels" <awels(a)redhat.com>
> > To: "Martin Perina" <mperina(a)redhat.com>
> > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>, devel(a)ovirt.org
> > Sent: Tuesday, June 2, 2015 4:16:31 AM
> > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag sent to FE
> >
> > On Monday, June 01, 2015 10:51:28 AM Martin Perina wrote:
> > > ----- Original Message -----
> > >
> > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > To: "Tomas Jelinek" <tjelinek(a)redhat.com>
> > > > Cc: "Liran Zelkha" <lzelkha(a)redhat.com>,
devel(a)ovirt.org
> > > > Sent: Monday, June 1, 2015 4:19:47 PM
> > > > Subject: Re: [ovirt-devel] hibernate's internal PersistentBag
sent to
> > > > FE
> > > >
> > > > On Monday, June 01, 2015 09:33:07 AM Tomas Jelinek wrote:
> > > > > ----- Original Message -----
> > > > >
> > > > > > From: "Alexander Wels" <awels(a)redhat.com>
> > > > > > To: devel(a)ovirt.org
> > > > > > Cc: "Tomas Jelinek" <tjelinek(a)redhat.com>,
"Liran Zelkha"
> > > > > > <lzelkha(a)redhat.com> Sent: Monday, June 1, 2015
3:16:34 PM
> > > > > > Subject: Re: [ovirt-devel] hibernate's internal
PersistentBag
> > > > > > sent
> > > > > > to
> > > > > > FE
> > > > > >
> > > > > > On Monday, June 01, 2015 09:08:50 AM Tomas Jelinek wrote:
> > > > > > > Hey all,
> > > > > > >
> > > > > > > since the org.ovirt.engine.core.common.job.Job/Step...
has been
> > > > > > > moved
> > > > > > > to
> > > > > > > use
> > > > > > > the JPA we have a problem on frontend. The problem is
that the
> > > > > > > @OneToMany
> > > > > > > annotations results in a List which is of type
PersistentBag.
> > > > > > > When
> > > > > > > we
> > > > > > > send
> > > > > > > this to Frontend it fails during deserialization. It
actually
> > > > > > > fails
> > > > > > > quite
> > > > > > > bad because the FE already has an ui-override of it
which is
> > > > > > > not
> > > > > > > correct
> > > > > > > resulting in a ton of NPEs in development mode.
> > > > > > >
> > > > > > > So, there are 2 nasty fixes I have made where none of
them
> > > > > > > should
> > > > > > > be
> > > > > > > merged
> > > > > > > but demonstrate the possibilities: 1: extend the FE to
be able
> > > > > > > to
> > > > > > > work
> > > > > > > with
> > > > > > > the PersistentBag
(https://gerrit.ovirt.org/#/c/41682/) not
> > > > > > > really
> > > > > > > good
> > > > > > > solution since the PersistenBag is an internal
Hibernate class
> > > > > > > which
> > > > > > > is
> > > > > > > really not meant to be passed around
> > > > > > >
> > > > > > > 2: fix on the backend to not send the PersistentBag
but an
> > > > > > > ArrayList.
> > > > > > > This
> > > > > > > is only a PoC fixed on a command we face the problem
> > > > > > > (https://gerrit.ovirt.org/#/c/41797/) Obviously this
is not
> > > > > > > going
> > > > > > > to
> > > > > > > work
> > > > > > > for other commands accessing the same Job nor for
other
> > > > > > > entities.
> > > > > > >
> > > > > > > So, the first option is generic but very very bad. The
second
> > > > > > > option
> > > > > > > should
> > > > > > > be used but not sure how to do this in a cheep way
(e.g.
> > > > > > > without
> > > > > > > using
> > > > > > > reflection to deep traverse everything sent back to
frontend
> > > > > > > checking
> > > > > > > if
> > > > > > > it
> > > > > > > does not have a PersistentBag in it.
> > > > > >
> > > > > > Tomas,
> > > > > >
> > > > > > Thanks, I was investigating the same issue, I noticed it
last
> > > > > > Friday
> > > > > > just
> > > > > > before leaving, so I was investigating the problem to see
what
> > > > > > was
> > > > > > going
> > > > > > on. You are right we should not be sending PersistentBag to
the
> > > > > > frontend
> > > > > > at all. So how about we do a combination of [1] and [2],
but
> > > > > > instead
> > > > > > of
> > > > > > delegating in [1] we actually simple throw an exception
stating
> > > > > > don't
> > > > > > sent PersistentBag to the front end. that way anyone
> > > > > > inadvertently
> > > > > > using
> > > > > > it will be notified immediately (since their code won't
work).
> > > > >
> > > > > Throwing the exception would help us in debugging but the main
> > > > > question
> > > > > is
> > > > > how will we make it work? Since we are planning to move more
and
> > > > > more
> > > > > to
> > > > > JPA so we will face this issue more and more often. Solving it
one
> > > > > by
> > > > > one
> > > > > on backend in each command is not going to work.
> > > >
> > > > How about something like this [1]? It appears to use some aspects to
> > > > translate
> > > > the hibernate internal classes into normal java util classes.
> > > >
> > > > [1] https://code.google.com/p/dehibernator/
> > >
> > > Well, not sure about that from performance point of view.
> > >
> > > If we really move away from GWT completely in 4.0, then implementing
> > > all
> > > hibernate inner collections in "uioverride" in a similar way as
Tomas
> > > did
> > > seems to me as the best solution from performance and "easiest to
> > > remove
> > > when not needed" point of view.
> > >
> >
> > I took some inspiration from that code and wrote my own fairly simple
> > hibernate persistent collection replacer, and I put the patch up here
> > [3].
> > It
> > still uses reflection so it is probably not the fastest thing ever
> > written.
> >
> > [3] https://gerrit.ovirt.org/#/c/41810/
> >
> > > > > > Alexander
> > > > > >
> > > > > > > Any better ideas?
> > > > > > > Thanx,
> > > > > > > Tomas
> > > > > > > _______________________________________________
> > > > > > > Devel mailing list
> > > > > > > Devel(a)ovirt.org
> > > > > > > http://lists.ovirt.org/mailman/listinfo/devel
> > > >
> > > > _______________________________________________
> > > > Devel mailing list
> > > > Devel(a)ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/devel
> >
> > _______________________________________________
> > Devel mailing list
> > Devel(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> >
> >
> >
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
3262
days inactive
3264
days old
20 comments
7 participants
participants (7)
-
Alexander Wels -
Liran Zelkha -
Martin Perina -
Michal Skrivanek -
Oved Ourfali -
Piotr Kliczewski -
Tomas Jelinek