Padding in a Pyspark Dataframe

Question

I have a Pyspark dataframe(Original Dataframe) having below data(all columns have string datatype):

  id           Value
   1             103
   2             1504
   3              1  

I need to create a new modified dataframe with padding in value column, so that length of this column should be 4 characters. If length is less than 4 characters, then add 0's in data as shown below:

  id             Value
   1             0103
   2             1504
   3             0001  

Can someone help me out? How can i achieve it using Pyspark dataframe? Any help will be appreciated.

Answer 1

You can use lpad from functions module,

from pyspark.sql.functions import lpad
>>> df.select('id',lpad(df['value'],4,'0').alias('value')).show()
+---+-----+
| id|value|
+---+-----+
|  1| 0103|
|  2| 1504|
|  3| 0001|
+---+-----+

Answer 2

Using PySpark lpad function in conjunction with withColumn:

import pyspark.sql.functions as F
dfNew = dfOrigin.withColumn('Value', F.lpad(dfOrigin['Value'], 4, '0'))