Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to save web page as text file [Python]

I would like to save a web page (all content) as a text file. (As if you did right click on webpage -> "Save Page As" -> "Save as text file" and not as html file)

I have tried using the following code:

import urllib2
url=''
page = urllib2.urlopen(url)
page_content = page.read()
file = open('file_text.txt', 'w')
f.write(page_content)
f.close()

My goal is to be able to save a whole text without html code. (for example i would like read "è" instead "&eacute")

like image 607
Skipper Avatar asked Feb 03 '16 00:02

Skipper


1 Answers

Have a look at html2text as mentioned elsewhere

import urllib2
import html2text
url=''
page = urllib2.urlopen(url)
html_content = page.read()
rendered_content = html2text.html2text(html_content)
file = open('file_text.txt', 'w')
file.write(rendered_content)
file.close()
like image 147
pnovotnak Avatar answered Sep 28 '22 22:09

pnovotnak