Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

add a different random number to every cell in a pandas dataframe

I need to add some 'noise' to my data, so I would like to add a different random number to every cell in my pandas dataframe. This code works, but seems unpythonic. Is there a better way?

import pandas as pd
import numpy as np
df = pd.DataFrame(0.0, index=[1,2,3,4,5], columns=list('ABC') )
print df
for x,line in df.iterrows():
  for col in df:
     line[col] = line[col] + (np.random.rand()-0.5)/1000.0
 print df
like image 449
TPM Avatar asked May 04 '17 15:05

TPM


People also ask

How do I add random numbers in pandas DataFrame?

Create Pandas Dataframe with Random float valuesUse the np. random. rand() to create a 2D numpy Array filled with random numbers from 0 to 1.

What is Nunique () in pandas?

Pandas DataFrame nunique() Method The nunique() method returns the number of unique values for each column. By specifying the column axis ( axis='columns' ), the nunique() method searches column-wise and returns the number of unique values for each row.

How do you keep unique values in pandas?

You can get unique values in column (multiple columns) from pandas DataFrame using unique() or Series. unique() functions. unique() from Series is used to get unique values from a single column and the other one is used to get from multiple columns.


2 Answers

df + np.random.rand(*df.shape) / 10000.0

OR

Let's use applymap:

df = pd.DataFrame(1.0, index=[1,2,3,4,5], columns=list('ABC') )

df.applymap(lambda x: x + np.random.rand()/10000.0)

output:

                                                   A  \
1  [[1.00006953418, 1.00009164785, 1.00003177706]...   
2  [[1.00007291245, 1.00004186046, 1.00006935173]...   
3  [[1.00000490127, 1.0000633115, 1.00004117181],...   
4  [[1.00007159622, 1.0000559506, 1.00007038891],...   
5  [[1.00000980335, 1.00004760836, 1.00004214422]...   

                                                   B  \
1  [[1.00000320322, 1.00006981682, 1.00008912557]...   
2  [[1.00007443802, 1.00009270815, 1.00007225764]...   
3  [[1.00001371778, 1.00001512412, 1.00007986851]...   
4  [[1.00005883343, 1.00007936509, 1.00009523334]...   
5  [[1.00009329606, 1.00003174878, 1.00006187704]...   

                                                   C  
1  [[1.00005894836, 1.00006592776, 1.0000171843],...  
2  [[1.00009085391, 1.00006606979, 1.00001755092]...  
3  [[1.00009736701, 1.00007240762, 1.00004558753]...  
4  [[1.00003981393, 1.00007505714, 1.00007209959]...  
5  [[1.0000031608, 1.00009372917, 1.00001960112],...  
like image 103
Scott Boston Avatar answered Sep 26 '22 01:09

Scott Boston


For nonzero data:

df + (np.random.rand(df.shape)-0.5)*0.001

OR

df + np.random.uniform(-0.01,0.01,(df.shape)))

For cases where your data frame contains zeros that you wish to keep as zero:

df * (1 + (np.random.rand(df.shape)-0.5)*0.001)

OR

df * (1 + np.random.uniform(-0.01,0.01,(df.shape)))

I think either of these should work, its a case of generating a same size "dataframe" (or perhaps array of arrays) as your existing df and adding it to your existing df (multiplying by 1 + random for cases where you wish zeros to remain zero). With the uniform function you can determine the scale of your noise by altering the 0.01 variable.

like image 43
tfcoe Avatar answered Sep 24 '22 01:09

tfcoe