Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Databricks - Displaying a Dataframe and printing a string

Tags:

I am trying to use one cell in databricks to display a dataframe and print some text underneath the display. I can't find anything on it on either the Databricks forum or here. It's a devilishly simple question so apologies if it is obvious.

myDF is a a pyspark.sql.dataframe

What I'm doing is:

myString = 'aasdf45'
print(myString)
display(myDF)

The output of the cell displays the DF, but the text isn't printed.

If I do this the other way around, printing the string after the display the result is still the same, showing the dataframe but no printed string:

myString = 'aasdf45'
display(myDF)
print(myString)

It seems like the behaviour is to display the DF over anything else. Is there a way I can show both? It's fine if not but it would be great to know.

Many thanks,

like image 582
Joseph Honeywood Avatar asked Oct 02 '18 15:10

Joseph Honeywood


1 Answers

Unfortunately, as things stand with Databricks, you cannot combine a table display with other content. You'll have to put the other content in a separate cell before/after the cell output.

The only alternative for single cell output is quite complicated:

  1. Collect the data you need to the driver using df.collect().
  2. Generate HTML for the content you need, e.g., build a TABLE from your data.
  3. Use displayHTML()
like image 118
Sim Avatar answered Sep 27 '22 20:09

Sim