Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to save back changes made to a HTML file using BeautifulSoup in Python?

I have the script below, which modifies href attributes in an HTML file (in the future, it will be a list of HTML files in a directory). Using BeautifulSoup I managed to access the tag values and modify them like I want, but I don't know how to save back the changes made to the file.

import os
import re
from bs4 import BeautifulSoup


htmlDoc = open('adding_computer_c.html',"r+")
soup = BeautifulSoup(htmlDoc)

replacements= [ ('_', '-'), ('../tasks/', prefixUrl), ('../concepts/', prefixUrl) ]

for link in soup.findAll('a', attrs={'href': re.compile("../")}):


    newlink=str(link)

    for k, v in replacements:

        newlink = newlink.replace(k, v)

    extrachars=newlink[newlink.find("."):newlink.find(">")]
    newlink=newlink.replace(extrachars,'')


    link=newlink
    print(link)
    ##How do I save the link I have modified back to the HTML file?

print(soup)##prints the original html tree

htmlDoc.close()
like image 558
PepeFloyd Avatar asked Jan 16 '13 22:01

PepeFloyd


People also ask

How do I edit HTML files using BeautifulSoup?

Step-by-step Approach:Step 1: First, import the libraries Beautiful Soup, os and re. Step 2: Now, remove the last segment of the path. Step 3: Then, open the HTML file in which you wish to make a change. Step 4: Moreover, parse the HTML file in Beautiful Soup.

Can I edit HTML with Python?

Python library is a stand-alone solution to edit HTML that doesn't rely on other software.

Can BeautifulSoup parse HTML?

The HTML content of the webpages can be parsed and scraped with Beautiful Soup.


1 Answers

newlink = link['href']
# .. make replacements
link['href'] = newlink # store it back

Now print(soup.prettify()) will show changed links. To save the changes to a file:

htmlDoc.close()

html = soup.prettify("utf-8")
with open("output.html", "wb") as file:
    file.write(html)

To preserve original character encoding of the document, you could use soup.original_encoding instead of "utf-8". See Encodings.

like image 72
jfs Avatar answered Oct 01 '22 03:10

jfs