How do I go to a random website? - python

Question

How to generate a random yet valid website link, regardless of languages. Actually, the more diverse the language of the website it generates, the better it is.

I've been doing it by using other people's script on their webpage, how can i not rely on these random site forwarding script and make my own?. I've been doing it as such:

import webbrowser
from random import choice
random_page_generator = ['http://www.randomwebsite.com/cgi-bin/random.pl',
                         'http://www.uroulette.com/visit']
webbrowser.open(choice(random_page_generator), new=2)

Answer 1

I've been doing it by using other people's script on their webpage, how can i not rely on these random site forwarding script and make my own?

There are two ways to do this:

• Create your own spider that amasses a huge collection of websites, and pick from that collection.
• Access some pre-existing collection of websites, and pick from that collection. For example, DMOZ/ODP lets you download their entire database;* Google used to have a customized random site URL;** etc.

There is no other way around it (short of randomly generating and testing valid strings of arbitrary characters, which would be a ridiculously bad idea).

Building a web spider for yourself can be a fun project. Link-driven scraping libraries like Scrapy can do a lot of the grunt work for you, leaving you to write the part you care about.

---

* Note that ODP is a pretty small database compared to something like Google's or Yahoo's, because it's primarily a human-edited collection of significant websites rather than an auto-generated collection of everything anyone has put on the web.

** Google's random site feature was driven by both popularity and your own search history. However, by feeding it an empty search history, you could remove that part of the equation. Anyway, I don't think it exists anymore.