Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

round() function not working for databricks-Python

I am trying to use the round() function in databricks to round some float values to 2 digits. However, the databricks python is not working like normal python.

Please help me with the reasons and solutions if any.

lis = [-12.1334, 12.23433, 1.2343, -104.444]
lis2 = [round(val,2)  for val in lis]
print(lis2)

TypeError: Invalid argument, not a string or column: -12.1334 of type <type 'float'>. For column literals, use 'lit', 'array', 'struct' or 'create_map' function.

Image Proof of Code

like image 602
munnahbaba Avatar asked Dec 21 '25 21:12

munnahbaba


1 Answers

This is only reproducible when you import the spark round function from the function module in spark.sql

The spark round function requires a string or a column. Which explains the error.

You can either alias the import such as import pyspark.sql.functions as F instead of from pyspark.sql.functions import *

You can get the origin round method this way.

import builtins
round = getattr(builtins, "round")

And then you can execute

lis = [-12.1334, 12.23433, 1.2343, -104.444]
lis2 = [round(val, 2) for val in lis]
print(lis2)
like image 60
BlueSheepToken Avatar answered Dec 24 '25 10:12

BlueSheepToken