How can I download Yahoo Groups?

Question

I want to download some Yahoo Groups (files, photos, messages, memberlist) and I've found these scripts:

• http://freshmeat.net/projects/grabyahoogroup/
• http://sourceforge.net/project/showfiles.php?group_id=62034

I've downloaded ActivePerl and the needed modules from CPAN (nothing fancy; they're very easy to find). I've managed to install them, but when I run the script I get an error after it tells me that I've successfully logged in: "Use of uninitialized value $cells in pattern match (m//) at yahoogroups_files.pl line 244, line 2."

I'm guessing that Yahoo changed the layout of the page or something, but I'm not able to update the script myself. I'm a newbie when it comes to Perl and understanding the way Yahoo generates the pages, I only know some basic C++. I want to mention that I'm not lazy, I'll try do fix it myself but I need your help: hints, advice, anything.

PS: I've contacted the author, but he isn't willing to update the scripts.

Answer 1

You would need knowledge in the following fields:

• use of an html parser

• http knowledge ( get/post/head )

• web scraping

I suggest you focus on WWW::Mechanize since it's capable of all these things ( and more )

EDIT: another solution ( that doesn't need programming ) , is this: login with your browser on yahoo groups, store the cookie, and then run wget , passing the stored cookie as a parameter. This way you'll get the task accomplished very fast.

Find your browser's cookies.txt file on your harddrive, and then call wget like this ( if I remember the commands correctly ) :

wget --load-cookies path_to_cookie_file -r -w 60 website

The full man page can be found here

EDIT2: Another option is to use WebDriver to automate firefox. You can use this article as a guide on how to accomplish this.