Simple queue for youtube-dl in the Linux shell

Question

youtube-dl is a Python script that allows one to download YouTube videos. It supports an option for batch downloads:

-a FILE, --batch-file=FILE
file containing URLs to download ('-' for stdin)

I want to setup some sort of queue so I can simply append URLs to a file and have youtube-dl process them. Currently, it does not remove files from the batch file. I see the option for '-' stdin and don't know if I can use this to my advantage.

In effect, I'd like to run youtube-dl as some form of daemon which will check the queue file and download the contained file names.

How can I do this?

Answer 1

The tail -f will not work because the script reads all the input at once.

It will work if you modify the script to perform a continuous read of the batch file.

Then simply run the script as:

% ./youtube-dl -a batch.txt -c

When you append some data into batch.txt, say:

% echo "http://www.youtube.com/watch?v=j9SgDoypXcI" >>batch.txt

The script will start downloading the appended video to the batch.

This is the patch you should apply to the latest version of "youtube-dl":

2278,2286d2277
<       while True:
<           batchurls = batchfd.readlines()
<           if not batchurls:
<               time.sleep(1)
<               continue
<           batchurls = [x.strip() for x in batchurls]
<           batchurls = [x for x in batchurls if len(x) > 0]
<           for bb in batchurls:
<               retcode = fd.download([bb])

Hope it helps, Happy video watching ;)

NOTE: Due to code restructuring this patch will no longer work. Would be interested to see if this could be added to the upstream code.

Answer 2

You might be able to get away with using tail -f to read from your file. It will not exit when it reaches end-of-file but will wait for more data to be appended to the file.

>video.queue  # erase and/or create queue file
tail -f video.queue | youtube-dl -a -

Since tail -f does not exit, youtube-dl should continue reading file names from stdin and never exit.