Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Storing a list of objects as csv in python

I am writing a program that scrapes a website and extracts the names and the links from a table. I store each name and respective link in an object and append it to a list of objects by running a for loop. I want to save this list as a csv file, but haven't been able to do so using the filewriter as it converts the whole object to str, which is not what I want. I would be really grateful if someone can guide me with this. Here is my code:

from selenium import webdriver
from bs4 import BeautifulSoup

class Player():
    def __init__(self):
        self.name = ""
        self.link = ""

player_list = []

driver = webdriver.PhantomJS(executable_path = r'C:\Users\sarim\Desktop\Scraper\phantomjs.exe')
driver.get('https://dak.gg/ranks/na/solo-fpp/rating')
pause = 0
lastHeight = driver.execute_script("return document.body.scrollHeight")
while True:
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    time.sleep(pause)
    newHeight = driver.execute_script("return document.body.scrollHeight")
    if newHeight == lastHeight:
        break
    lastHeight = newHeight

html_doc = driver.page_source
soup = BeautifulSoup(html_doc, 'lxml')

players = soup.find('table', class_ = 'list rating')
player_names = players.find_all('td', class_ = 'nick')

add_link = "https://dak.gg"

for i in player_names[0:3]:
    newPlay = Player()
    n = i.find('a')
    l = add_link + n['href']
    newPlay.link = l
    newPlay.name = n.text.strip()
    #print newPlay.name
    #print newPlay.link
    player_list.append(newPlay)

#print players_names[1]
driver.quit()
like image 748
Sarim Zia Khan Avatar asked Dec 23 '22 06:12

Sarim Zia Khan


1 Answers

Try something like this

import csv
with open('data.csv', 'w',) as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(['Name', 'Link'])
    for player in player_list:
        writer.writerow([player.name, player.link])
like image 74
Sunitha Avatar answered Dec 25 '22 19:12

Sunitha