Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do i get my Beautiful soup output data to a text file?

How do i get my Beautiful soup output data to a text file?
Here's the code;

import urllib2

from bs4 import BeautifulSoup

url = urllib2.urlopen("http://link").read()

soup = BeautifulSoup(url)

file = open("parseddata.txt", "wb")

for line in soup.find_all('a', attrs={'class': 'book-title-link'}):

 print (line.get('href'))

 file.write(line.get('href'))

 file.flush()

 file.close()
like image 623
max10 Avatar asked Dec 04 '25 15:12

max10


1 Answers

file.close should be called once (after the for loop):

import urllib2
from bs4 import BeautifulSoup

url = urllib2.urlopen("http://link").read()
soup = BeautifulSoup(url)
file = open("parseddata.txt", "wb")
for line in soup.find_all('a', attrs={'class': 'book-title-link'}):
    href = line.get('href')
    print href
    if href:
        file.write(href + '\n')
file.close()

UPDATE you can use href=True to avoid if statement. In addition to it, using with statement, you don't need to close the file object manually:

import urllib2
from bs4 import BeautifulSoup


content = urllib2.urlopen("http://link").read()
soup = BeautifulSoup(content)

with open('parseddata.txt', 'wb') as f:
    for a in soup.find_all('a', attrs={'class': 'book-title-link'}, href=True):
        print a['href']
        f.write(a['href'] + '\n')
like image 52
falsetru Avatar answered Dec 06 '25 03:12

falsetru



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!