How to implement 3-way conference call video chat with WebRTC Native Code for Android?

Question

I'm trying to implement 3-way video chat inside an Android app using the WebRTC Native Code package for Android (i.e. not using a WebView). I've written a signalling server using node.js and used the Gottox socket.io java client library inside the client app to connect to the server, exchange SDP packets and establish a 2-way video chat connection.

However now I'm having problems going beyond that to a 3-way call. The AppRTCDemo app that comes with the WebRTC native code package demonstrates 2-way calls only (if a 3rd party attempts to join a room a "room full" message is returned).

According to this answer (which doesn't relate to Android specifically), I'm supposed to do it by creating multiple PeerConnections, so each chat participant will connect to the 2 other participants.

However, when I create more than one PeerConnectionClient (a Java class which wraps a PeerConection, which is implemented on the native side in libjingle_peerconnection_so.so), there is an exception thrown from inside the library resulting from a conflict with both of them trying to access the camera:

E/VideoCapturerAndroid(21170): startCapture failed E/VideoCapturerAndroid(21170): java.lang.RuntimeException: Fail to connect to camera service E/VideoCapturerAndroid(21170):  at android.hardware.Camera.native_setup(Native Method) E/VideoCapturerAndroid(21170):  at android.hardware.Camera.<init>(Camera.java:548) E/VideoCapturerAndroid(21170):  at android.hardware.Camera.open(Camera.java:389) E/VideoCapturerAndroid(21170):  at org.webrtc.VideoCapturerAndroid.startCaptureOnCameraThread(VideoCapturerAndroid.java:528) E/VideoCapturerAndroid(21170):  at org.webrtc.VideoCapturerAndroid.access$11(VideoCapturerAndroid.java:520) E/VideoCapturerAndroid(21170):  at org.webrtc.VideoCapturerAndroid$6.run(VideoCapturerAndroid.java:514) E/VideoCapturerAndroid(21170):  at android.os.Handler.handleCallback(Handler.java:733) E/VideoCapturerAndroid(21170):  at android.os.Handler.dispatchMessage(Handler.java:95) E/VideoCapturerAndroid(21170):  at android.os.Looper.loop(Looper.java:136) E/VideoCapturerAndroid(21170):  at org.webrtc.VideoCapturerAndroid$CameraThread.run(VideoCapturerAndroid.java:484) 

This happens when initializing the local client even before attempting to establish a connection so it's not related to node.js, socket.io or any of the signalling server stuff.

How do I get multiple PeerConnections to share the camera so that I can send the same video to more than one peer?

One idea I had was to implement some kind of singleton camera class to replace VideoCapturerAndroid that could be shared between multiple connections, but I'm not even sure that would work and I'd like to know if there is a way to do 3-way calls using the API before I start hacking around inside the library.

Is it possible and if so, how?

Update:

I tried sharing a VideoCapturerAndroid object between multiple PeerConnectionClients, creating it for the first connection only and passing it into the initialization function for the subsequent ones, but that resulted in this "Capturer can only be taken once!" exception when creating a second VideoTrack from the VideoCapturer object for the second peer connection:

E/AndroidRuntime(18956): FATAL EXCEPTION: Thread-1397 E/AndroidRuntime(18956): java.lang.RuntimeException: Capturer can only be taken once! E/AndroidRuntime(18956):    at org.webrtc.VideoCapturer.takeNativeVideoCapturer(VideoCapturer.java:52) E/AndroidRuntime(18956):    at org.webrtc.PeerConnectionFactory.createVideoSource(PeerConnectionFactory.java:113) E/AndroidRuntime(18956):    at com.example.rtcapp.PeerConnectionClient.createVideoTrack(PeerConnectionClient.java:720) E/AndroidRuntime(18956):    at com.example.rtcapp.PeerConnectionClient.createPeerConnectionInternal(PeerConnectionClient.java:482) E/AndroidRuntime(18956):    at com.example.rtcapp.PeerConnectionClient.access$20(PeerConnectionClient.java:433) E/AndroidRuntime(18956):    at com.example.rtcapp.PeerConnectionClient$2.run(PeerConnectionClient.java:280) E/AndroidRuntime(18956):    at android.os.Handler.handleCallback(Handler.java:733) E/AndroidRuntime(18956):    at android.os.Handler.dispatchMessage(Handler.java:95) E/AndroidRuntime(18956):    at android.os.Looper.loop(Looper.java:136) E/AndroidRuntime(18956):    at com.example.rtcapp.LooperExecutor.run(LooperExecutor.java:56) 

Attempting to share the VideoTrack object between PeerConnectionClients resulted in this error from the native code:

E/libjingle(19884): Local fingerprint does not match identity. E/libjingle(19884): P2PTransportChannel::Connect: The ice_ufrag_ and the ice_pwd_ are not set. E/libjingle(19884): Local fingerprint does not match identity. E/libjingle(19884): Failed to set local offer sdp: Failed to push down transport description: Local fingerprint does not match identity. 

Sharing the MediaStream between PeerConnectionClients results in the app abruptly closing, without any error message appearing in the Logcat.

Answer 1

The problem you are having is that PeerConnectionClient is not a wrapper around PeerConnection it contains a PeerConnection.

I noticed this question wasn't answered so I wanted to see if I could help out a bit. I looked into the source code and PeerConnectionClient is very much hard coded for a single remote peer. You would need to create a collection of PeerConnection objects rather then this line:

private PeerConnection peerConnection; 

If you look around a bit more you would notice that it gets a bit more complicated then that.

The mediaStream logic in createPeerConnectionInternal should only be done once and you need to share the stream between your PeerConnection objects like this:

peerConnection.addStream(mediaStream); 

You can consult the WebRTC spec or take a look at this stackoverflow question to confirm that the PeerConnection type was designed to only handle one peer. It is also somewhat vaguely implied here.

So you only maintain one mediaStream object:

private MediaStream mediaStream; 

So again the main idea is one MediaStream object and as many PeerConnection objects as you have peers you want to connect to. So you will not be using multiple PeerConnectionClient objects, but rather modify the single PeerConnectionClient to encapsulate the multi-client handling. If you do want to go with a design of multiple PeerConnectionClient objects for whatever reason you would just have to abstract the media stream logic (and any support types that should only be created once) out of it.

You will also need to maintain multiple remote video tracks rather then the existing one:

private VideoTrack remoteVideoTrack; 

You would obviously only care to render the one local camera and create multiple renderers for the remote connections.

I hope this is enough information to get you back on track.