Unresolved reference while trying to import col from pyspark.sql.functions in python 3.5

Question

Refer to the post here: Spark structured streaming with python I would like to import 'col' in python 3.5

from pyspark.sql.functions import col

However I got an error saying unresolved reference to col. I've installed pyspark library, so just wondering has the 'col' been removed from pyspark library? And how can I import the 'col' then.

Answer 1

Try installing 'pyspark-stubs', I had the same problem in PyCharm and by doing it I resolved it.

Answer 2

It turns out to be IntelliJ IDEA's problem. Even though it shows unresolved reference, my program still runs without any problem in the command line.