WebRTC for realtime scaling

Question

I'm looking for a cheapo solution to realtime scaling for many users in 1 channel.

I'm using sockjs but scaling is pretty annoying when talking about really large numbers.

I'm thinking about using webrtc to decrease cost with p2p. Instead of the server connecting to all users it would connect to only limited number of users who would then distribute their data to the p2p network through webrtc. Is this sensible? What's the easiest way to implement?

The information is not private and a few seconds <5s of latency is acceptable.

Answer 1

Yes, that is what webRTC is for: exchanging data between browsers without a server. To make this work you might want to think out some management plan on the server that manages the connections (as one client needs to tell the other client 'hey I am here, connect to me..'), and thinking about what pieces of the data are on which client, when a client needs to connect to someone else.

As a tip: I made a server in nodejs using websockets (and nodejs plugin: 'ws') to communicate between clients until an RTC is set up so they can stream audio. It is really easy to do all this stuff, but a pain to refine.

As I can read from your question, you have no experience using webRTC. Just to give you a head start, this is the right order of things that need to happen to make an RTC:

Client 1                        Server       Client 2
Create an RTC object
Create offer
set localdescription = offer                 Create RTC object                   
send offer ------------------->       ------>set remote description to offer
                                             Create answer
                                             local description = answer
set remote description<--------       <------send answer

This goes both directions:
onicecandidate send ---------->       ------>set ICE candidate

Connection done!

For exchanging this connection data my advice is to go with websockets. Both clients open a websocket, and whenever one client sends something, you can take the websocket connection (it's an object) from the other client and send things. Using XHR's you can only let the browser connect, ask for data, and if the data is not there retry in x seconds.

sum: Using and setting up webRTC for peer to peer connections is fairly easy, but managing who shall connect to who is gonna be a lot of trouble.

Edit: My idea would be that the first client connects to the server, and receives it's data either by an XHR or websockets or something like that. If you want browser compatibility, you might want to go with socket.io, but that wouldn't really matter since only chrome and firefox support webRTC (afaik). Then you simply connect from your browser to the sever. If you do plan on having multiple exchanges at the same a session ID would be fairly handy, so that you can just exchange that with the people that need to download it.

On the server side, the websockets returns an object 'containing' the current connection. So if client 1 would connect, you store that inside an object with the ID we made. If client 2 would connect, you can store that too. Then you can just take the websocket object from the first client and do .send('your message').

Now for real how I would do it. I will use (ws) to determine the connection over a websocket, (http) for http request and (rtc) for webRTC. pc is your peer connection object: window.dc = new RTCDataChannel(ICEServers);. sdp means SessionDescriptionProtocol,

1. A client connects to your server and downloads the neccessarry files (http) (I mean the webpage, scripts and css, not the file you want to share.)
2. The client requests a new session. (ws)

3. the server creates an instance of the session inside an object.

   var sessID = Math.random().toString(36).substring(12, 16);
   sessions[sessID] = {};

4. You send this session ID to the client (ws), so it can send this to other people (using mail etc). The other user connects to the server(http)(ws), but does not request an ID, it sends it.

5. When the server receives this it sends both the first client and this client a message containing that both parties are ready. The sending client creates a new webRTC object, and creates a new offer (it stores this offer in dc.setLocalOffer(sdp)). It sends this to the server (ws) and the server sends this to client 2 (ws). Now the client stores it using dc.setRemoteDescription(sdp), and creates an answer. (dc.createAnswer()), sets this answer(dc.setLocalDescription(sdp). This answer is sent to client one. Now you can use the datachannel.

I don't know how the datachannel exacly works, as I have only been working with PeerConnection, which is especially for audio and video streams. You can use this to find out how to make the connection. The code for this is in my repo. There is a lot more code inside as this is for a 'profielwerkstuk' (some workpiece for school, which I made with a friend). The things that are interesting for you to look at are server.js (this is the nodejs server, used for exchanging the sdp and ICE candidates). The webpage is in htdocs/mp.html (not really interesting), and the piece of javascript code for doing this all is on htdocs/scripts/rtc.js.

For more information you might want to look at this example, and at the explainments here

There is already 'something you want', and it uses a lot of code to make this possible. Also please note that stackoverflow is to ask questions, not to ask for ready made code. If you want that, take a look at carreers 2.0, and find someone who wants to do this for you.

Edit 2: Now I see your answer, I think the best way to go is to store all the connections inside an array inside your session object, and go for connecting client 1 -> 2, 2-> 3, 3-> 4 etc. But well then you have issues with parts and such. Think about torrenting, where you connect to multiple people and download small parts from each. It is really hard, yes, and I don't think someone has already done something so big (apart from sharefest). If you really want to make this you will need to do the most yourself. Take some time to think about how you would solve this, and use stackoverflow (or other information sites) to look for how to setup rtc and such.

Answer 2

Completely sensible.

WebRTC is low latency P2P networking in the browser. If the user's browser has DataChannel support then you could use a library like WebTorrent https://github.com/feross/webtorrent.

Also have a look at Sharefest's implementation: https://www.sharefest.me/ it's very similar to the idea you're looking for.

To get started, have a look at the HTML5 Rocks WebRTC tutorial: http://www.html5rocks.com/en/tutorials/webrtc/basics/. Also the SimpleWebRTC library is quite handy: http://simplewebrtc.com/