Flutter: How to get the microphone's raw audio signal into memory in real time?

Question

Well, the title says it all. The idea is to monitor ambient noise. As soon as a certain pattern is detected, the audio signal should be recorded to a file. The difficulty is that the recorded file should start a few seconds before the pattern is detected. Hence the audio signal is needed in memory to be able to "go back" a few seconds. Do you have any ideas how to get the raw audio input into memory in real time?

Answer 1

you can use the flutter sound plugin to get the microphone's raw audio signal into memory in real-time. reference link is: https://pub.dev/packages/flutter_sound and demo example is here https://github.com/dooboolab/flutter_sound/blob/master/example/lib/demo_util/demo3_body.dart

I have one example is

Widget _buildRecorder(Track track) {
    return Padding(
        padding: const EdgeInsets.all(8.0),
        child: RecorderPlaybackController(
            child: Column(
          children: [
            Left("Recorder"),
            SoundRecorderUI(track),
            Left("Recording Playback"),
            SoundPlayerUI.fromTrack(
              track,
              enabled: false,
              showTitle: true,
              audioFocus: true
                  ? AudioFocus.requestFocusAndDuckOthers
                  : AudioFocus.requestFocusAndDuckOthers,
            ),
          ],
        )));
  }

}

Answer 2

The sound_stream package can help you stream microphone audio as a Stream<Uint8List> in PCM format. Assuming that you have, 16000 samples per second, you can use this to create a stream that yields n second audio chunks:

import 'dart:math'; // for min
import 'dart:typed_data'; // for Uint8List
import 'package:sound_stream/sound_stream.dart';

Stream<Uint8List> audioChunks([int seconds = 3]) async* {
  final stream = RecorderStream();
  await stream.initialize();
  stream.start();

  BytesBuilder buffer = BytesBuilder();
  int filled = 0;
  int max = seconds * 16000 * 2; // pcm 16bit = 2bytes / sample

  await for (final chunk in stream.audioStream) {
    int cl = min(max-filled, chunk.length);
    buffer.add(chunk.sublist(0, cl));
    filled += cl;
    if (filled == max) {
      yield buffer.toBytes();
      filled = 0;
      buffer.clear();
    }
    if (chunk.length > cl) {
      buffer.add(chunk.sublist(cl));
      filled += chunk.length - cl;
    }
  }
}