How to write a program that download all videos from a web page and sort them according to date created? [closed]

Question

I'm new to iOS development. In order to learn iOS developement, I want to download all WDCC videos from apple developer website from 2013 to 2016. I can do this manually but the process would be tedious and repetitive. It should be possible to write a program where it can search through the WDCC web page that contain all the download links to the videos, download them to my local compute, classify and put them into different folders (e.g. according to year).

But due to my inexperience, I have this idea but I dont know where to start. It would be great if someone could give me a general steps and idea as to how this could be done.

Answer 1

First, I would use youtube-dl to download the videos. It's a good general-purpose video downloader and I see it works on the WWDC videos. You can install it here.

Next, you need to script it. Notice that the WWDC videos follow a certain naming scheme: https://developer.apple.com/videos/play/wwdcYEAR/NUMBER/.

To automate this, you can use bash. Save this script somewhere as download.sh and then run chmod +x download.sh on it to make it executable.

for year in {2013..2016}; do
    for number in {100..999}; do
        youtube-dl https://developer.apple.com/videos/play/wwdc$year/$number/
        sleep 5; # Or however long you need to wait
    done;
done;

Note that this script makes some huge assumptions, trying to download session videos 100 through 999. Clearly, most of these do not exist: If you know the numbers you want, you can do for number in {100, 102, 104, 106..121}; do in order to download videos 100, 102, 104, and 106 through 121.

As well, you may run into throttling issues. I added a sleep 5 in there to have it wait 5 seconds after each video: You may find that's not enough. You can adjust the parameters as needed, though!

Hope that helps.