How to get Audio for encoding using Xuggler

Question

I'm writing an application that records the screen and audio. While the screen recording works perfectly, I'm having difficulty in getting the raw audio using the JDK libraries. Here's the code:

try {
            // Now, we're going to loop
            long startTime = System.nanoTime();

            System.out.println("Encoding Image.....");
            while (!Thread.currentThread().isInterrupted()) {
                // take the screen shot
                BufferedImage screen = robot.createScreenCapture(screenBounds);


                // convert to the right image type
                BufferedImage bgrScreen = convertToType(screen,
                        BufferedImage.TYPE_3BYTE_BGR);

                // encode the image
                writer.encodeVideo(0, bgrScreen, System.nanoTime()
                        - startTime, TimeUnit.NANOSECONDS);

                /* Need to get audio here and then encode using xuggler. Something like 

                    WaveData wd = new WaveData();

                    TargetDataLine line;
                    AudioInputStream aus = new AudioInputStream(line);

                    short[] samples = getSourceSamples();
                       writer.encodeAudio(0, samples); */


                if (timeCreation < 10) {
                    timeCreation = getGMTTime();
                }
                // sleep for framerate milliseconds
                try {
                    Thread.sleep((long) (1000 / FRAME_RATE.getDouble()));
                } catch (Exception ex) {
                    System.err.println("stopping....");
                    break;
                }

            }
            // Finally we tell the writer to close and write the trailer if
            // needed
        } finally {
            writer.close();
        }

This page has some pseudo code like

while(haveMoreAudio())
 {
   short[] samples = getSourceSamples();
   writer.encodeAudio(0, samples);
 }

but what exactly should I do for getSourceSamples()?

Also, a bonus question - is it possible to choose from multiple microphones in this approach?

See also: Xuggler encoding and muxing

Answer 1

Try this:

// Pick a format. Need 16 bits, the rest can be set to anything
// It is better to enumerate the formats that the system supports, because getLine() can error out with any particular format
AudioFormat audioFormat = new AudioFormat(44100.0F, 16, 2, true, false); 

// Get default TargetDataLine with that format
DataLine.Info dataLineInfo = new DataLine.Info( TargetDataLine.class, audioFormat );
TargetDataLine line = (TargetDataLine) AudioSystem.getLine(dataLineInfo);

// Open and start capturing audio    
line.open(audioFormat, line.getBufferSize());
line.start();

while (true) {
    // read as raw bytes
    byte[] audioBytes = new byte[ line.getBufferSize() / 2 ]; // best size?
    int numBytesRead = 0;
    numBytesRead =  line.read(audioBytes, 0, audioBytes.length);

    // convert to signed shorts representing samples
    int numSamplesRead = numBytesRead / 2;
    short[] audioSamples = new short[ numSamplesRead ];
    if (format.isBigEndian()) {
        for (int i = 0; i < numSamplesRead; i++) {
            audioSamples[i] = (short)((audioBytes[2*i] << 8) | audioBytes[2*i + 1]);
        }
    }
    else {
        for (int i = 0; i < numSamplesRead; i++) {
            audioSamples[i] = (short)((audioBytes[2*i + 1] << 8) | audioBytes[2*i]);
        }
    }

    // use audioSamples in Xuggler etc
}

To pick a microphone, you'd probably have to do this:

Mixer.Info[] mixerInfo = AudioSystem.getMixerInfo();
// Look through and select a mixer here, different mixers should be different inputs
int selectedMixerIndex = 0;
Mixer mixer = AudioSystem.getMixer(mixerInfo[ selectedMixerIndex ]);
TargetDataLine line = (TargetDataLine) mixer.getLine(dataLineInfo);

I think it's possible that multiple microphones will show up in one mixer as different source data lines. In that case you'd have to open them and call dataLine.getControl(FloatControl.Type.MASTER_GAIN).setValue( volume ); to turn them on and off.

See: WaveData.java

Sound wave from TargetDataLine

How to set volume of a SourceDataLine in Java