How can I pipe a Python process' output in Python?

Question

I'm writing a program that downloads videos from YouTube, using youtube-dl.

I used to call youtube-dl with subprocess:

import subprocess

p = subprocess.Popen([command], \
    stdout=subprocess.PIPE, \
    stderr=subprocess.STDOUT, \
    universal_newlines = True)

Then, I would read the process' output by calling:

for line in iter(p.stdout.readline, ""):
    hide_some_stuff_using_regex()
    show_some_stuff_using_regex()

However, I prefer using youtube-dl as a Python class. So I'm now doing this:

from youtube_dl import YoutubeDL as youtube_dl

options = {"restrictfilenames": True, \
           "progress_with_newline": True}

ydl = youtube_dl(options)
ydl.download([url])

The code works, but I'm having a hard time finding out, how to pipe youtube-dl's output. Note that I want to use parts of youtube-dl's output to print in real-time, so redirecting sys.stdout to a custom output stream will not work, as I still need sys.stdout to print.

Can you help me?

Answer 1

Specifically for youtube-dl, you can set a logger object, like in the advanced example in the documentation:

from youtube_dl import YoutubeDL


class MyLogger(object):
    def debug(self, msg):
        print('debug information: %r' % msg)

    def warning(self, msg):
        print('warning: %r' % msg)

    def error(self, msg):
        print('error: %r' % msg)


options = {
    "restrictfilenames": True,
    "progress_with_newline": True,
    "logger": MyLogger(),
}

url = 'http://www.youtube.com/watch?v=BaW_jenozKc'
with YoutubeDL(options) as ydl:
    ydl.download([url])