Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Data Scraping: appending data doesn't work

I have an Excel file (Compay, Start_Date, End_Date) from which I want to read the components and find the stock data available from the NSE website and save that data to a CSV file.

So I am trying this code:

cf = pd.read_csv('Company.csv')
cf['START_DT']=pd.to_datetime(cf['START_DT'])
cf['END_DT']=pd.to_datetime(cf['END_DT'])
cf

OUTPUT - 
    COMPANY START_DT    END_DT
0   SBIN    2014-01-01  2018-01-01
1   PNB     2014-01-01  2018-01-01
2   INFY    2014-01-01  2018-01-01

for index,row in cf.iterrows():
    start_dt=row['START_DT']
    end_dt=row['END_DT']
    data = get_history(symbol=row['COMPANY'], start=start_dt, end=end_dt)
    print(data)
data.to_csv('data.csv', sep=',')

Howevery, the data stored is only the data of the last company. I have tried to append the data using a dataframe but it does not work neither.

How can I fix this?

like image 759
Patit Pawan Barik Avatar asked Jan 30 '26 08:01

Patit Pawan Barik


1 Answers

Here is necessary create list of all DataFrames via append and concat together:

dfs = []
for index,row in cf.iterrows():
    start_dt=row['START_DT']
    end_dt=row['END_DT']
    data = get_history(symbol=row['COMPANY'], start=start_dt, end=end_dt)
    dfs.append(data)

df = pd.concat(dfs)
df.to_csv('data.csv')
like image 135
jezrael Avatar answered Jan 31 '26 21:01

jezrael