[Engine-devel] Showing stacktrace info in GUI error dialog

Wed Jan 15 14:52:50 UTC 2014

Hi guys, I discussed this with Martin and here are my conclusions.

Any server-side exception (thrown in context of JBoss AS runtime) should always
get logged into server.log or engine.log depending on logger configuration. The
fact that our mysterious InvocationTargetException didn't appear in any logfile
indicates it might have originated from client code - after receiving GWT RPC
response, something goes wrong during RPC payload processing.

Since both BZs mentioned below [2,3] cannot be reliably reproduced, we looked
at GWT RPC code. Martin couldn't trigger this exception from any server side code
[command/query execution, Backend EJB, GenericApiGWTServiceImpl/RpcServlet] so
the only logical place was to look at client code doing RPC payload processing.

The symptoms of this mysterious InvocationTargetException are following:
- GWT RPC response arrives with HTTP 200 (OK) status [*]
- nothing gets logged in server.log or engine.log
- still, something goes wrong and GUI shows error dialog due to GWT RPC
  AsyncCallback#onFailure execution

[*] Frontend#failureEventHandler reaches "else" block so "caught" is not
    StatusCodeException (thrown when server responds with non-OK status)

Overall, symptoms indicate problem with RPC payload serialization. Martin has
just verified that throwing exception in custom field serializer's "serialize"
method yields the exact symptoms as described above. So GWT client, in an
attempt to make RPC call, invokes the custom serializer which fails; GUI shows
an error dialog but no RPC call is made due to the error.

Since we only have few custom serializers, most classes rely on default GWT RPC
serialization rules. This means it's very likely that there was a problem during
serialization of a specific type, which didn't adhere to GWT RPC serialization
rules..

..which brings us to a patch merged recently to guard against missing no-arg
constructors in shared classes: http://gerrit.ovirt.org/#/c/20274/

(After Martin backported this patch to Engine codebase corresponding to versions
reported in BZs, he indeed found lots of classes with missing no-arg c-tors.)

To summarize: we think both BZs [2,3] were caused by GWT RPC serialization
problems, which shouldn't occur anymore due to no-arg c-tor check. I cannot
say for sure, because both these BZs cannot be reliably reproduced..

As a precaution, we could take patches from this BZ:

  https://bugzilla.redhat.com/1015683

and backport & verify them against both BZs [2,3].

Regards,
Vojtech

----- Original Message -----
> From: "Martin Perina" <mperina at redhat.com>
> To: "engine-devel" <engine-devel at ovirt.org>
> Cc: "Einav Cohen" <ecohen at redhat.com>, "users" <users at ovirt.org>, "Greg Sheremeta" <gshereme at redhat.com>, "Vojtech
> Szocs" <vszocs at redhat.com>, "Barak Azulay" <bazulay at redhat.com>
> Sent: Wednesday, January 15, 2014 7:02:19 AM
> Subject: Re: [Engine-devel] Showing stacktrace info in GUI error dialog
> 
> Hi,
> 
> I would like to correct a few things in this case.
> 
> The reason for patch [1] was to identify, from which part of engine code
> InvocationTargetException came from. (more details are at [2] and [3]).
> In both cases this exception was not logged in engine.log or server.log
> 
> Trying to verify this patch I found out this:
> 
> 1) If this exception was raised during command/query execution, it would be
> part
>    VdsReturnValue and shown in UI in Operation Canceled dialog. And what's
>    most
>    important, the exception would be definitely logged in engine.log
> 
> 2) If this exception was raised on backend side outside of command/query
> execution
>    (I simulated this by throwing RuntimeException inside
>    Backend.runAction/runQuery)
>    it would be shown in UI with HTTP status code (500 in this case) and also
>    would
>    be logged in engine.log
> 
> 3) If this exception was raised somewhere servlet part of frontend code
> (GenericApiGWTServiceImpl)
>    it would also be shown in UI with HTTP status code (500 in this case) and
>    also
>    would be logged in server.log
> 
> 4) For all other cases there is shown dialog in UI with
>     'Error while executing action: A Request to the Server failed: XXX'
>    where XXX is the exception message. Patch [1] adds stacktrace to this
>    dialog so
>    we will be able to identify origin of those rare exceptions.
> 
> So the exception was raised somewhere else and since bugs [2],[3] wasn't been
> successfully
> reproduced yet, it's the only way how to find out origin of the exception.
> 
> And I should also note, that I have no intention to show stacktrace in UI for
> "normal"
> exceptions described in 1), 2) or 3)
> 
> Thanks
> 
> Martin
> 
> 
> [1] http://gerrit.ovirt.org/23096
> [2] http://bugzilla.redhat.com/1039248
> [3] http://bugzilla.redhat.com/1034437
> 
> 
> 
> ----- Original Message -----
> > From: "Greg Sheremeta" <gshereme at redhat.com>
> > To: "Einav Cohen" <ecohen at redhat.com>
> > Cc: "engine-devel" <engine-devel at ovirt.org>, "users" <users at ovirt.org>,
> > "Martin Perina" <mperina at redhat.com>
> > Sent: Tuesday, January 14, 2014 6:35:53 PM
> > Subject: Re: [Engine-devel] Showing stacktrace info in GUI error dialog
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Einav Cohen" <ecohen at redhat.com>
> > > To: "engine-devel" <engine-devel at ovirt.org>, "users" <users at ovirt.org>,
> > > "Martin Perina" <mperina at redhat.com>
> > > Sent: Tuesday, January 14, 2014 11:23:34 AM
> > > Subject: [Engine-devel] Showing stacktrace info in GUI error dialog
> > > 
> > > Hi, this is about patch [1] - showing server-side-exception
> > > stacktrace info in GUI error dialog.
> > > 
> > > a couple of notes about this patch:
> > > 
> > > (1) I find it a little strange that server-side exceptions
> > > are not logged in the server side and are / will be displayed
> > > only on the client-side - this needs to be fixed first.
> > > 
> > +1.
> > 
> > > (2) once everything is logged on the server side, we can
> > > debate whether it makes sense to put the full exception
> > > stack-trace *also* in the GUI error pop-up.
> > > I think that it is not completely necessary (since if there
> > > is a server side exception, the user is likely to go to the
> > > server side log anyway to get the full picture / context), but
> > > if it is (and I'd like to hear your opinion about that) - we
> > > shouldn't show the entire exception in the dialog once the dialog
> > > is displayed - that's too much information to absorb.
> > > We should show whatever we are showing today, and also something
> > > like a "More Details" collapsible section (collapsed by default)
> > > that, when expanded, displays the full exception details.
> > > 
> > > comments are welcome.
> > 
> > I personally appreciate web apps that allow me to see an error (or even
> > expose an entire log file), but do so gracefully -- hide it under a option,
> > use a "More Details" option like you mention, etc.
> > 
> > I think saving a person the step of ssh'ing to a server and digging for a
> > log
> > file is a nice-to-have.
> > 
> > Greg
> > 
> > > 
> > > ----
> > > Thanks,
> > > Einav
> > > 
> > > [1] http://gerrit.ovirt.org/#/c/23096/
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > 
> 