Django build video website similar to YouTube

Question

I am trying to build a video upload, view, sharing website using

Softwares :

Django 1.11.8

Django Rest Framework 3.7.3

Python 3.6.3

Backend Database MySQL 5.7.19

HTTP server : Apache 2.4.27

mod_wsgi 4.5.22

WAMP server 3.1.0

HTML5

CSS3

Bootstrap 3.3.7

Javascript ECMAScript 5

Server Hardware Info:

Window 2012 server Xeon processor 32 GB RAM

240 GB Intel 730 Series SSD

4 SAMSUNG 850 PRO 2.5" SSD 1TB (Total 4TB)

Memcached 1.45 for caching

BACKEND': 'django.core.cache.backends.memcached.MemcachedCache

I am worried about the performance and scalability.

I want to share my approach and please correct me If I am wrong or you can suggest me better way.

I already have default user database on Django.contrib.auth.model>>User and will be use same users information to upload, likes, views and other interactions with videos

01. Create Model.py (this is sample)

    from django.db import models

    from django.contrib.auth.models import User


    class Channel(models.Model):
        name = models.CharField(max_length=100)
        subscribers = models.ManyToManyField(User, related_name='+', blank=True)
        creation_time = models.DateTimeField(auto_now_add=True)
        modified_time = models.DateTimeField(auto_now=True)


    class Subscription(models.Model):
        user = models.ForeignKey(User, on_delete=models.CASCADE, related_name='+')
        channel = models.ForeignKey(Channel, on_delete=models.CASCADE)
        notifications = models.BooleanField(default=True)


    class Playlist(models.Model):
        name = models.CharField(max_length=100)
        video = models.ManyToManyField('Video')
        user = models.ForeignKey(User, on_delete=models.CASCADE)


    class Category(models.Model):
        name = models.CharField(max_length=100)


    class ChannelSettings(models.Model):
        channel = models.ForeignKey(Channel, on_delete=models.CASCADE)
        verified = models.BooleanField(default=False)

02 Create View.py

To render Html templates frames. such as video view page, channel page, playlist page, history page, search page I will not use Django template tags to populate any information.

03 Create api.py

To create Rest API using django rest framework with this I will get and put the video information, metadata, likes, views and other updates.

04 JavaScript

Read information through Ajax and render page dynamically. For example get list of watch history and render with thumbnails. etc in html page.

Please suggest me best way to build the video upload page with video upload progression, auto thumbnail creation and important video compression.

General Information about my video upload website:

Approximate videos up to now : 5000+ (on average time 30 min)

Users : 400 +

Original video size - 4K, 2K and minimum 1280*720

Is this approach good enough for performance and scale?

Answer 1

Here are some suggestions:

• I rather go with a Linux server like Ubuntu.

• Playlists have more than one video so I think you missed there. ( you're using ForeignKey instead of ManyToMany)

• when it comes to streaming high-quality videos like 4K, you should focus on the server that you're going to serve the files with. I mean making a website to just show some content containing a player for videos won't be much challenging, while file server can be because it's the file server that does the heavy lifting.

• I suggest you make some background tasks to handle files. like moving them to another server (or upload videos to another server directly) and if you need to create multiple resolutions out of a video file like what youtube does, you need another server to do that and a whole bunch of codes to handle that for you.

• I suggest you write 2 different projects connecting to one single database (or you can make an API on your main project instead) which one of them handles the user stuff and rendering the website for you and the other one to handle static files.

• I think Nginx is a good candidate for your file server.

• Make some limitations so users/bots won't be able to upload too much video at the same time. That can cause heavy loads at the time.

• Make a queue to handle jobs, don't do them all at once or just after they got uploaded, wait for the right time to handle files so you won't crash your server(s) on peek times.

• If you have the budget, you can go with JW player.

• I would choose PostgreSQL over MySQL. it's also been suggested by Django and you have some features that aren't available on MySQL.

These are some suggestion to consider if you are thinking about scaling. But the numbers you mentioned can be achieved with less. Those numbers aren't that big so it's not that hard to handle.

I will write more if anything came to mind later.