Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Replace text without escaping in BeautifulSoup

I would like to wrap some words that are not already links with anchor links in BeautifulSoup. I use this to achieve it:

from bs4 import BeautifulSoup
import re

text = ''' replace this string '''

soup = BeautifulSoup(text)
pattern = 'replace'

for txt in soup.findAll(text=True):
    if re.search(pattern,txt,re.I) and txt.parent.name != 'a':
        newtext = re.sub(r'(%s)' % pattern,
                         r'<a href="#\1">\1</a>',
                         txt)
        txt.replaceWith(newtext)
print(soup)

Which unfortunately returns

<html><body><p>&lt;a href="#replace"&gt;replace&lt;/a&gt; this string </p></body></html>

Whereas I am looking for:

<html><body><p><a href="#replace">replace</a> this string </p></body></html>

Is there a way in which I can tell BeautifulSoup not to escape the link elements?

A simple regex to replace will not do here because I will eventually not only have one pattern that I want to replace but multiple. This is why I decided to use BeautifulSoup to exclude everything that already is a link.

like image 462
k-nut Avatar asked Jun 07 '15 10:06

k-nut


1 Answers

You need to create new tag using new_tag use insert_after to insert part of your text after your newly created a tag.

for txt in soup.find_all(text=True):
    if re.search(pattern, txt, re.I) and txt.parent.name != 'a':
        newtag = soup.new_tag('a')
        newtag.attrs['href'] = "#{}".format(pattern)
        newtag.string = pattern
        txt.replace_with(newtag)
        newtag.insert_after(txt.replace(pattern, ""))
like image 188
styvane Avatar answered Nov 01 '22 23:11

styvane