How to read audio data from a 'MediaStream' object in a C++ addon

Question

After sweating blood and tears I've finally managed to set up a Node C++ addon and shove a web-platform standard MediaStream object into one of its C++ methods for good. For compatibility across different V8 and Node.js versions, I'm using Native Abstractions for Node.js (nan):

addon.cc

NAN_METHOD(SetStream)
{
    Nan::HandleScope scope;
    v8::Local<v8::Object> mediaStream = info[0]->ToObject();
}

addon.js

setStream(new MediaStream());

For what it's worth, this works correctly (i.e. it does not obliterate the renderer process on sight), and I can verify the presence of the MediaStream object, e.g. by returning its constructor name from the C++ method:

addon.cc

info.GetReturnValue().Set(mediaStream->GetConstructorName());

When called from JavaScript through setStream, this would return the string MediaStream, so the object is definitely there. I can also return the mediaStream object itself and everything will work correctly, so it's indeed the object I need.

So, how would I read audio data (i.e. audio samples) from this MediaStream object in C++? As a sidenote, the actual data read (and processing) would be done in a separate std::thread.

---

Bounty Update

I understand this would be sort of easier/possible if I were compiling Electron and/or Chromium myself, but I'd rather not get involved in that maintenance hell.

I was wondering if it would be possible without doing that, and as far as my research goes, I'm convinced I need 2 things to get this done:

1. The relevant header files, for which I believe blink public should be adequate
2. A chromium/blink library file (?), to resolve external symbols, similarly to the node.dylib file

Also, as I said, I believe I could compile chromium/blink myself and then I would have this lib file, but that would be a maintenance hell with Electron. With this in mind, I believe this question ultimately comes down to a C++ linking question. Is there any other approach to do what I'm looking for?

Edit

ScriptProcessorNode is not an option in my case, as its performance makes it nearly unusable in production. This would require to process audio samples on the ui/main thread, which is absolutely insane.

Edit 2

AudioWorklets have been available in Electron for some time now, which, unlike the ScriptProcessorNode (or worse, the AnalyzerNode), is low-latency and very reliable for true C++ backed audio processing even in real time.

If someone wants to go ahead and write an AudioWorklet-based answer, I'll gladly accept, but beware: it's a very advanced field and a very deep rabbit hole, with countless obstacles to get through even before a very simple, general-purpose pass-through prototype (especially so because currently in Electron, Atomics-synced, buffered cross-thread audio processing is required to pull this off because https://github.com/electron/electron/issues/22503 -- although getting a native C++ addon into one audio renderer thread, let alone multiple threads at the same time, is probably equally as challenging).

Answer 1

The MediaStream header is part of Blink's renderer modules, and it's not obvious to me how you could retrieve this from nan plugin.

So, instead let's look at what you do have, namely a v8::Object. I believe that v8::Object exposes all the functionality you need, it has:

• GetPropertyNames()
• Get(context, index)
• Set(context, key, value)
• Has(context, key)

Unless you really need a strictly defined interface, why not avoid the issue altogether and just use the dynamic type that you already have?

For getting audio data out specifically, you would need to call getAudioTracks() on the v8::Object, which probably looks something like this?

Note: I don't think you need a context, v8 seems to be happy with it being empty: v8/src/api/api.cc

Should look something like this, plus some massaging of types in and out of v8.

v8::MaybeLocal<v8::Value> get_audio_tracks = mediaStream->Get("getAudioTracks");
// Maybe needs to be v8::Object or array?
if (!get_audio_tracks.IsEmpty()) {
    v8::Local<v8::Value> audio_tracks = get_audio_tracks.ToLocalChecked()();
}