Filtering and counting negative/positive values from a Spark dataframe using pyspark?

Question

I have no clue how to filter for positive or negative values within a column using pyspark, can you help?

I have a spark dataframe with 10MM+ rows and 50+ columns and need to count the times the values in one specific column are equal or less than 0.

Thanks in advance.

Answer 1

For the column you want to target you can simply filter the dataframe for when the value is <= 0 and count the number of rows that meet the criteria.

import pyspark.sql.functions as func

df.filter(func.col("colname") <= 0).count()