Online audio/video conference system in html5

Question

I'm trying to figure out if HTMl5 is suited for the client part of an online conference system. The client must be capable to:
1. display live video provided by the server, using the video tag.
2. Similar for the live audio, using audio tag.
3. The system supports text messaging also. Here we can use websockets
4. There is also a desktop sharing feature. For this kind of data stream I was also thinking to websockets. But this is binary data, it can be encoded in base64 before sending. So in the html5 client, it has to be decoded, processed (it is a proprietary protocol) and using a canvas object (?!) draw it to the screen.

Can the webapp process this amount of data in the same time ?
Is it HTML5 prepared for this ?

Answer 1

• Can webapps process this amount off data? Yes

• Is HTML5 prepared for this? Not yet, but soon

These are all areas that HTML5 is working to address. However, some of the working groups are farther along than others and the features have differing levels of implementation in browsers. Ericsson is doing a lot in this area. They have a patched version of webkit that enables enough of these technologies to do usable video/audio conferencing.

In terms of desktop sharing, noVNC (VNC client in a browser) demonstrates that this is possible. noVNC (disclaimer: I wrote noVNC) does full RFB/VNC decode and render in the browser using Javascript and Canvas. It uses WebSockets to send and receive the data and base64 encode/decodes over the wire since WebSockets doesn't support binary data yet. It uses a WebSockets to TCP proxy websockify to communicate with the VNC servers. It performs quite well.

Here are linked so some of the relevant standards work:

• HTML5 index

• Full web-apps standard

• Canvas

• video and audio tags

• Media capture

• Media capture API

• Device tag/element

• WebSockets API

• Current WebSockets protocol in Chrome/Safari

• All WebSockets protocol drafts

• ArrayBuffer/Typed Arrays

• stream API

• File API

The best place to see what the status of various HTML5 related technologies is: http://caniuse.com