python del vs pandas drop

Question

I know it might be old debate, but out of pandas.drop and python del function which is better in terms of performance over large dataset?

I am learning machine learning using python 3 and not sure which one to use. My data is in pandas data frame format. But python del function is in built-in function for python.

Answer 1

Summarizing a few points about functionality:

• drop operates on both columns and rows; del operates on column only.
• drop can operate on multiple items at a time; del operates only on one at a time.
• drop can operate in-place or return a copy; del is an in-place operation only.

The documentation at https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.drop.html has more details on drop's features.

Answer 2

Using randomly generated data of about 1.6 GB, it appears that df.drop is faster than del, especially over multiple columns:

df = pd.DataFrame(np.random.rand(20000,10000))
t_1 = time.time()
df.drop(labels=[2,4,1000], inplace=True)
t_2 = time.time()
print(t_2 - t_1)

0.9118959903717041

Compared to:

df = pd.DataFrame(np.random.rand(20000,10000))
t_3 = time.time()
del df[2]
del df[4]
del df[1000]
t_4 = time.time()
print(t_4 - t_3)

4.052732944488525

@Inder's comparison is not quite the same since it doesn't use inplace=True.