Audio not clear while streaming between a java class and android activity

Question

I have an android activity, which connects to a java class and sends data packets to it in form of sockets. The class receives the sound packets and throws them to PC speakers. The code is working excellently, but there is a constant jitter/ interruption while the sound is played in PC speakers.

The android activity:

public class SendActivity extends Activity {
    private Button startButton, stopButton;

    public byte[] buffer;
    public static DatagramSocket socket;
    private int port = 50005;
    AudioRecord recorder;

    private int sampleRate = 8000;
    @SuppressWarnings("deprecation")
    private int channelConfig = AudioFormat.CHANNEL_IN_MONO;
    private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;
    int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig,
            audioFormat);
    private boolean status = true;


    int bufferSizeInBytes;
    int bufferSizeInShorts;
      int shortsRead;
      short audioBuffer[];

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_send);

        startButton = (Button) findViewById(R.id.start_button);
        stopButton = (Button) findViewById(R.id.stop_button);

        startButton.setOnClickListener(new View.OnClickListener() {

            @Override
            public void onClick(View v) {

                status = true;
                startStreaming();


            }

        });

        stopButton.setOnClickListener(new View.OnClickListener() {

            @Override
            public void onClick(View v) {

                status = false;
                recorder.release();
                Log.d("VS", "Recorder released");

            }

        });

        minBufSize += 5120;
        System.out.println("minBufSize: " + minBufSize);
    }


    public void startStreaming() {

        Thread streamThread = new Thread(new Runnable() {

            @Override
            public void run() {
                try {

                    DatagramSocket socket = new DatagramSocket();
                    Log.d("VS", "Socket Created");

                    byte[] buffer = new byte[minBufSize];

                    Log.d("VS", "Buffer created of size " + minBufSize);
                    DatagramPacket packet;
//machine's IP
                    final InetAddress destination = InetAddress
                            .getByName("192.168.1.20");
                    Log.d("VS", "Address retrieved");

                    recorder = new AudioRecord(MediaRecorder.AudioSource.VOICE_RECOGNITION,
                            sampleRate, channelConfig, audioFormat,
                            minBufSize * 10);
                    Log.d("VS", "Recorder initialized");

                    recorder.startRecording();

                    while (status == true) {

                        // reading data from MIC into buffer
                        minBufSize = recorder.read(buffer, 0, buffer.length);

                        // putting buffer in the packet
                        packet = new DatagramPacket(buffer, buffer.length,
                                destination, port);

                        socket.send(packet);
                        System.out.println("MinBufferSize: " + minBufSize);

                    }

                } catch (UnknownHostException e) {
                    Log.e("VS", "UnknownHostException");
                } catch (IOException e) {
                    e.printStackTrace();
                    Log.e("VS", "IOException");
                }
            }

        });
        streamThread.start();
    }


}

The android layout:

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:paddingBottom="@dimen/activity_vertical_margin"
    android:paddingLeft="@dimen/activity_horizontal_margin"
    android:paddingRight="@dimen/activity_horizontal_margin"
    android:paddingTop="@dimen/activity_vertical_margin"
    tools:context=".SendActivity" >



    <Button
        android:id="@+id/stop_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignBaseline="@+id/start_button"
        android:layout_alignBottom="@+id/start_button"
        android:layout_toRightOf="@+id/start_button"
        android:text="Stop" />

    <Button
        android:id="@+id/start_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_alignParentLeft="true"
        android:layout_alignParentTop="true"
        android:layout_marginLeft="79dp"
        android:layout_marginTop="163dp"
        android:text="Start" />

</RelativeLayout>

Android Manifest:

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.audiostreamsample"
    android:versionCode="1"
    android:versionName="1.0" >

    <uses-sdk
        android:minSdkVersion="8"
        android:targetSdkVersion="17" />

    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" >
    </uses-permission>
    <uses-permission android:name="android.permission.INTERNET" >
    </uses-permission>
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" >
    </uses-permission>
    <uses-permission android:name="android.permission.READ_PHONE_STATE" >
    </uses-permission>
    <uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
    <uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
    <uses-permission android:name="android.permission.GET_ACCOUNTS" />
    <uses-permission android:name="android.permission.CALL_PHONE" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />

    <application
        android:allowBackup="true"
        android:icon="@drawable/ic_launcher"
        android:label="@string/app_name"
        android:theme="@style/AppTheme" >
        <activity
            android:name="com.example.audiostreamsample.SendActivity"
            android:label="@string/app_name" >
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>

The class to receive the data packets and throw them to the PC speakers:

class Server {

AudioInputStream audioInputStream;
static AudioInputStream ais;
static AudioFormat format;
static boolean status = true;
static int port = 50005;
static int sampleRate = 8000;



public static void main(String args[]) throws Exception {


    DatagramSocket serverSocket = new DatagramSocket(50005);

    /**
     * Formula for lag = (byte_size/sample_rate)*2
     * Byte size 9728 will produce ~ 0.45 seconds of lag. Voice slightly broken.
     * Byte size 1400 will produce ~ 0.06 seconds of lag. Voice extremely broken.
     * Byte size 4000 will produce ~ 0.18 seconds of lag. Voice slightly more broken then 9728.
     */

    byte[] receiveData = new byte[5000];

    format = new AudioFormat(sampleRate, 16, 1, true, false);

    while (status == true) {
        DatagramPacket receivePacket = new DatagramPacket(receiveData,
                receiveData.length);

        serverSocket.receive(receivePacket);

        ByteArrayInputStream baiss = new ByteArrayInputStream(
                receivePacket.getData());

        ais = new AudioInputStream(baiss, format, receivePacket.getLength());
        toSpeaker(receivePacket.getData());

    }



}

public static void toSpeaker(byte soundbytes[]) {
    try {

        DataLine.Info dataLineInfo = new DataLine.Info(SourceDataLine.class, format);
        SourceDataLine sourceDataLine = (SourceDataLine) AudioSystem.getLine(dataLineInfo);

        sourceDataLine.open(format);

        FloatControl volumeControl = (FloatControl) sourceDataLine.getControl(FloatControl.Type.MASTER_GAIN);
        volumeControl.setValue(6.0206f);

        sourceDataLine.start();
        sourceDataLine.open(format);

        sourceDataLine.start();

        System.out.println("format? :" + sourceDataLine.getFormat());

        sourceDataLine.write(soundbytes, 0, soundbytes.length);
        System.out.println(soundbytes.toString());
        sourceDataLine.drain();
        sourceDataLine.close();
    } catch (Exception e) {
        System.out.println("Not working in speakers...");
        e.printStackTrace();
    }
}
}

If you want to test the app in your IDE, then simply create two different projects, one for the android app and one for the server class.

In the android app just add the IP of your machine and run the app on a device, the mobile and the computer should belong to the same network. Please execute the server class as a java application.

The jitter will be prominent and irritating but the voices will be more or less clear. Please suggest me what to do to get a clearer output.

Answer 1

You need to have some coded support for actual streaming. There's a little more to consider than just sending datagrams and hoping for the best.

Real networks are not perfect.

• Delay: packets take time
• Jitter : the time a packet takes in flight is not constant
• Dropped packets: sometimes they don't make it.
• Reordering : sometimes packets arrive in a different order to the sending.

You should read up on simple media streaming protocols like RTP and perhaps use a library that provides RTP to both ends. RTP commonly sits atop UDP.

TCP streaming for audio can be less helpful than UDP/RTP , as you'd have to turn off Nagling.

You will at a minimum need a small buffer at the receiver end to prevent buffer empty causing sound dropouts.