Stream android screen

Question

I am trying to fallow Truiton ScreenCapture example, in order to record the device screen using MediaProjection

When saving the recording localy it works

    mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
    mMediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);

    String localFilePath = getLocalFilePath();
    mMediaRecorder.setOutputFile(localFilePath);

    mMediaRecorder.setVideoSize(DISPLAY_WIDTH, DISPLAY_HEIGHT);
    mMediaRecorder.setVideoEncoder(MediaRecorder.VideoEncoder.MPEG_4_SP);
    mMediaRecorder.setVideoFrameRate(30);
    mMediaRecorder.prepare();

How ever when changing to work with FileDescriptor it's not

    mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
    mMediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);

    String hostname = "10.26.100.18";
    int port = 2007;
    Socket socket = new Socket(InetAddress.getByName(hostname), port);
    ParcelFileDescriptor fileDescriptor = ParcelFileDescriptor.fromSocket(socket);
    LocalServerSocket localServerSocket = new LocalServerSocket(fileDescriptor.getFileDescriptor());

    mMediaRecorder.setOutputFile(localServerSocket.getFileDescriptor());

    mMediaRecorder.setVideoSize(DISPLAY_WIDTH, DISPLAY_HEIGHT);
    mMediaRecorder.setVideoEncoder(MediaRecorder.VideoEncoder.MPEG_4_SP);
    mMediaRecorder.setVideoFrameRate(30);
    mMediaRecorder.prepare();

If not using the LocalServerSocket, then mMediaRecorder.prepare() throw exception, now that I am using it, get exception in the below method in mMediaRecorder.getSurface()

private VirtualDisplay createVirtualDisplay() {
    return mMediaProjection.createVirtualDisplay("MainActivity",
            DISPLAY_WIDTH, DISPLAY_HEIGHT, mScreenDensity,
            DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
            mMediaRecorder.getSurface(), null /*Callbacks*/, null
            /*Handler*/);
}

The exception

Caused by: java.lang.IllegalStateException: failed to get surface
   at android.media.MediaRecorder.getSurface(Native Method)
   at com.truiton.screencapture.MainActivity$override.createVirtualDisplay(MainActivity.java:172)
   at com.truiton.screencapture.MainActivity$override.onActivityResult(MainActivity.java:133)
   at com.truiton.screencapture.MainActivity$override.access$dispatch(MainActivity.java)
   at com.truiton.screencapture.MainActivity.onActivityResult(MainActivity.java:0)
   at android.app.Activity.dispatchActivityResult(Activity.java:6456)
   at android.app.ActivityThread.deliverResults(ActivityThread.java:3695)
   at android.app.ActivityThread.handleSendResult(ActivityThread.java:3742) 
   at android.app.ActivityThread.-wrap16(ActivityThread.java) 
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1393) 
   at android.os.Handler.dispatchMessage(Handler.java:102) 
   at android.os.Looper.loop(Looper.java:148) 
   at android.app.ActivityThread.main(ActivityThread.java:5417) 
   at java.lang.reflect.Method.invoke(Native Method) 
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726) 
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616) 

This is my Java server, I get socket after mMediaRecorder.prepare() is called and it stuck on inputStream.read as eccpected. The exception in Android happands when I call mMediaRecorder.start()

    final byte[] buffer = new byte[1024];
    try {
        ServerSocket serverSocket = new ServerSocket(2007);
        while (!serverSocket.isClosed()) {
            Socket socket = serverSocket.accept();
            File videoFile = createEmptyVideoFile();
            FileOutputStream outputStream = new FileOutputStream(videoFile);
            InputStream inputStream = socket.getInputStream();
            int length = inputStream.read(buffer);
            while (length != -1) {
                outputStream.write(buffer, 0, length);
                length = inputStream.read(buffer);
            }
        }
    } catch (IOException e) {
        e.printStackTrace();
    }

Answer 1

You have to use LocalServerSocket, below is what partially worked for me, I have a MediaStreamer class which extends MediaRecorder.

public class MediaStreamer extends MediaRecorder {

    private LocalServerSocket localServerSocket = null;
    private LocalSocket receiver, sender = null;

    public void prepare() throws IllegalStateException,IOException {

        receiver = new LocalSocket();
        try {
            localServerSocket = new LocalServerSocket("<your_socket_addr>");
            receiver.connect(new LocalSocketAddress("<your_socket_addr>"));
            receiver.setReceiveBufferSize(4096);
            receiver.setSendBufferSize(4096);
            sender = localServerSocket.accept();
            sender.setReceiveBufferSize(4096);
            sender.setSendBufferSize(4096); 
        } catch (IOException e1) {
            throw new IOException("Can't create local socket !");
        }

        setOutputFile(sender.getFileDescriptor());

        try {
            super.prepare();
        } catch (Exception e) {
            closeSockets();
            throw e;
        }           
    }

    public InputStream getInputStream() {

        InputStream out = null;

        try {
            out = receiver.getInputStream();
        } catch (IOException e) {
        }

        return out;

    }


    public void stop() {
        closeSockets();
        super.stop();
    }

    private void closeSockets() {
        if (localServerSocket !=null) {
            try {
                localServerSocket.close();
                sender.close();
                receiver.close();
            }
            catch (IOException e) {

            }
            localServerSocket = null; 
            sender = null; 
            receiver = null;
        }
    }
}

For Recording

video = new MediaStreamer();
video.reset();

video.setVideoSource(MediaRecorder.VideoSource.DEFAULT);
video.setPreviewDisplay(holder.getSurface());
video.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
video.setVideoFrameRate(VideoConstants.frameRate);
video.setVideoEncodingBitRate(VideoConstants.bitRate*1000);
video.setVideoSize(VideoConstants.resolationX, VideoConstants.resolationY);
video.setVideoEncoder(MediaRecorder.VideoEncoder.MPEG_4_SP);

try {
   video.prepare();
} catch (IOException e) {
   e.printStackTrace();
}
video.start();

---

But the main problem is mp4 is not very easy to stream . The basic problem is that MP4 is not live, streamable format, so even though data is captured via the socket, the crucial file headers which are normally written at the conclusion of an audio or video capture, are missing (because sockets are not seekeable like local files) - hence the unplayable data (And so, why it works fine when saved as local files, is understandable).

There is no easy way to perform post-processing on the data to manually add the file headers. So the solution is either don't use MP4 as the recording format, or, write a packetiser similar to what is used in the Spydroid project

Hope this helps !