Python RoboBrowser Automated Search Function

Question

What is wrong with this code? It works with duckduckgo.com but not google, wikipedia, or yahoo. And yes I am changing the id according to the specific website.

import re
from robobrowser import RoboBrowser

browser = RoboBrowser() 
browser.open("https://en.wikipedia.org/wiki/Wikipedia")

# Must find the proper id in the html
form = browser.get_form(id = "searchInput")
form

form["searchval"].value = "Beethoven Opus 131"
browser.submit_form(form)

links = browser.get_links()

for link in links:
    print(link)

print("Le Fin.")

I get the following error every single time (except duckduckgo.com)

line 16, in <module>
    form["searchval"].value = "Beethoven Opus 131"
TypeError: 'NoneType' object is not subscriptable

Why am I getting the NoneType error here? I know the variable was not defined previously, but not with Duckduckgo either. Please help.

Answer 1

First of all, you are locating the form incorrectly. The element with id="searchInput" is an input element while you need a form element - it has id="searchform".

Also, since there are 2 submit buttons, you need to let robobrowser which one to use:

form = browser.get_form(id="searchform")
form["search"].value = "Beethoven Opus 131"
browser.submit_form(form, submit=form.submit_fields['go'])

Worked for me.