How to overcome an HTMLUnit ScriptException?

Question

I got a problem with a line of code that probably triggers some js function e couse an exception, how can i fix this?

box.setText(link.toString()); client.waitForBackgroundJavaScriptStartingBefore(10000); box.dblClick(); //this line cause the exception  Exception in thread "main" ======= EXCEPTION START ======== EcmaError: lineNumber=[0] column=[0] lineSource=[function () {] name=[ReferenceError] sourceName=[onclick event for HtmlDivision[<div class="_119 stat_elem focus_target mtm mbl _5bsm _6dh _51z6" id="u_0_k" data-location="maincolumn" onclick="Bootloader.loadComponents(&quot;ComposerXControllerBootload&quot;, emptyFunction);">] in https://www.facebook.com/?_fb_noscript=1] message=[ReferenceError: "Bootloader" is not defined.] com.gargoylesoftware.htmlunit.ScriptException: ReferenceError: "Bootloader" is not defined.     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:684)     at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:602)     at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507)     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:616)     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:591)     at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptFunctionIfPossible(HtmlPage.java:985)     at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeEventHandler(EventListenersContainer.java:210)     at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:230)     at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:804)     at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:738)     at com.gargoylesoftware.htmlunit.html.HtmlElement$1.run(HtmlElement.java:869)     at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:602)     at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507)     at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:874)     at com.gargoylesoftware.htmlunit.html.HtmlElement.doClickFireClickEvent(HtmlElement.java:1311)     at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1253)     at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1205)     at com.gargoylesoftware.htmlunit.html.HtmlElement.dblClick(HtmlElement.java:1351)     at com.gargoylesoftware.htmlunit.html.HtmlElement.dblClick(HtmlElement.java:1326)     at prototype.Profile.postLinkOnWall(Profile.java:225)     at html.Log.findNext(Log.java:150)     at prototype.Prtp.main(Prtp.java:49) Caused by: net.sourceforge.htmlunit.corejs.javascript.EcmaError: ReferenceError: "Bootloader" is not defined.     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3603)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3587)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.notFoundError(ScriptRuntime.java:3657)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.nameOrFunction(ScriptRuntime.java:1749)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.name(ScriptRuntime.java:1690)     at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1622)     at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)     at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)     at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:405)     at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3031)     at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:103)     at com.gargoylesoftware.htmlunit.javascript.host.EventHandler.call(EventHandler.java:81)     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:609)     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:669)     ... 21 more Enclosed exception:  net.sourceforge.htmlunit.corejs.javascript.EcmaError: ReferenceError: "Bootloader" is not defined.     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3603)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3587)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.notFoundError(ScriptRuntime.java:3657)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.nameOrFunction(ScriptRuntime.java:1749)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.name(ScriptRuntime.java:1690)     at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1622)     at script.onclick(onclick event for HtmlDivision[<div class="_119 stat_elem focus_target mtm mbl _5bsm _6dh _51z6" id="u_0_k" data-location="maincolumn" onclick="Bootloader.loadComponents(&quot;ComposerXControllerBootload&quot;, emptyFunction);">] in https://www.facebook.com/?_fb_noscript=1)     at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)     at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)     at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:405)     at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309)     at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3031)     at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:103)     at com.gargoylesoftware.htmlunit.javascript.host.EventHandler.call(EventHandler.java:81)     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:609)     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:669)     at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:602)     at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507)     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:616)     at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:591)     at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptFunctionIfPossible(HtmlPage.java:985)     at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeEventHandler(EventListenersContainer.java:210)     at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:230)     at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:804)     at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:738)     at com.gargoylesoftware.htmlunit.html.HtmlElement$1.run(HtmlElement.java:869)     at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:602)     at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507)     at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:874)     at com.gargoylesoftware.htmlunit.html.HtmlElement.doClickFireClickEvent(HtmlElement.java:1311)     at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1253)     at com.gargoylesoftware.htmlunit.html.HtmlElement.click(HtmlElement.java:1205)     at com.gargoylesoftware.htmlunit.html.HtmlElement.dblClick(HtmlElement.java:1351)     at com.gargoylesoftware.htmlunit.html.HtmlElement.dblClick(HtmlElement.java:1326)     at prototype.Profile.postLinkOnWall(Profile.java:225)     at html.Log.findNext(Log.java:150)     at prototype.Prtp.main(Prtp.java:49) == CALLING JAVASCRIPT == function () {     [native code, arity=0] }  ======= EXCEPTION END ======== 

The box where I write, acts on a normal browser a reformatting function which is not performed using HtmlUnit so I tried to force it with a dbclick().

Answer 1

HtmlUnit does not play well with JavaScript. It will frequently throw errors complaining about variables or functions not defined.

In that sense, real life browsers (FireFox, Internet Explorer, Chrome, etc) are much more flexible. That means that they will allow syntactically incorrect pieces of HTML and JavaScript (eg: not defining functions or not ending HTML tags).

HtmlUnit expects everything to be (almost) perfect. Although, it will fix some missing ending HTML tags, in general, it expects the code in the pages not to contain any kind of error. Furthermore, even if everything looks correct HtmlUnit might even complain.

Some items for you to think about are:

• The most important one is switching between different BrowserVersions. You can set them when creating the WebClient object. Internet Explorer (ironically) has proven to give me the best results when it comes to interpreting JavaScript
• Make sure your HTML and JavaScript code are both correct
• Avoid using of complex libraries (jQuery seems to be properly supported)
• Try to use non-minimized versions of libraries
• If you happen to be using jQuery (or other similar libraries) avoid complex jQuery methods (eg: dynamically adding events to elements)

Of course, those comments would apply if you have control over the source code you're fetching from the server. Sometimes, this is not the case. In this situation your hands are even more tied.

One option, would be to suppress the exception with:

webClient.getOptions().setThrowExceptionOnScriptError(false); 

Although, this will get you through the exception won't correct any JavaScript error. That means that if the JS piece of code that is throwing this exception happens to be crucial in your logic, I mean, you absolutely depend on the result of the execution of that code then you can not let HtmlUnit handle your JS. If this happens to be the result of an AJAX request then you can issue the request yourself manually instead of letting HtmlUnit do so.

On the other hand, if the JS code that is giving you trouble is not critical in your logic, I mean, it might just be hiding an element or changing a color that you don't care about, then suppressing the exception would be the way to go.

There aren't many options left.