Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to use sparks implicit conversion (e.g. $) in IntelliJ debugger evaluate expression

When debugging Spark/Scala code with IntelliJ, using e.g. df.select($"mycol") does not work in the evaluate expression window, while df.select(col("mycol")) works fine (but needs code change):

enter image description here

It says :

Error during generated code invocation: com.intellij.debugger.engine.evaluation.EvaluateException: Error evaluating method : 'invoke': Method threw 'java.lang.NoSuchFieldError' exception.: Error evaluating method : 'invoke': Method threw 'java.lang.NoSuchFieldError' exception.

Strangely, it seems to work sometimes, especially if the $ is already part of an existing expression in the code which I mark to evaluate. If I write arbitrary expressions (code-fragments), it fails consistently

EDIT: even repeating import spark.implicts._ in the code-fragment window does not help

like image 300
Raphael Roth Avatar asked May 28 '21 14:05

Raphael Roth


People also ask

How do I evaluate an expression in Intellij debugger?

Evaluate a complex expression in the editorClick Run | Debugging Actions | Quick Evaluate Expression Ctrl+Alt+F8 . Alternatively, hold Alt and click the selection.

What is the shortcut for evaluating the value of an expression?

Open the Evaluate dialog in one of the following ways: Press Shift+F9 . Select Evaluate Expression from the context menu. Choose Run | Debugging Actions | Evaluate Expression from the main menu.

How do I debug an application in Intellij?

Run the program in debug modeClick the Run icon in the gutter, then select Modify Run Configuration. Enter arguments in the Program arguments field. Click the Run button near the main method. From the menu, select Debug.


1 Answers

Try this workaround:

import spark.implicits._
$""
df.select($"diff").show()

before

after

like image 119
Dean Xu Avatar answered Sep 22 '22 03:09

Dean Xu