Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get float array of samples from audio file

I’m developing a UWP application ( for Windows 10) which works with audio data. It receives samples buffer at the start in the form of a float array of samples, which items are changing from -1f to 1f. Earlier I used NAudio.dll 1.8.0 that gives all necessary functionality. Worked with WaveFileReader, waveBuffer.FloatBuffer, WaveFileWriter classes. However, when I finished this app and tried to build Release version, got this error: ILT0042: Arrays of pointer types are not currently supported: 'System.Int32*[]'.

I’ve tried to solve it:

  1. https://forums.xamarin.com/discussion/73169/uwp-10-build-fail-arrays-of-pointer-types-error

There is advice to remove the link to .dll, but I need it.

  1. I’ve tried to install NAudio the same version using Manage NuGet Packages, but WaveFileReader, WaveFileWriter is not available.

  2. In NAudio developer’s answer (How to store a .wav file in Windows 10 with NAudio) I’ve read about using AudioGraph, but I can build float array of samples only in the realtime playback, but I need get the full samples to pack right after the audio file uploading. Example of getting samples during the recording process or playback: https://docs.microsoft.com/ru-ru/windows/uwp/audio-video-camera/audio-graphs

That’s why I need help: how to get FloatBuffer for working with samples after audio file uploading? For example, for building audio waves or calculation for audio effects applying.

Thank you in advance.


  1. I’ve tried to use FileStream and BitConverter.ToSingle(), however, I had a different result compared to NAudio. In other words, I’m still looking for a solution.

     private float[] GetBufferArray()
     {
         string _path = ApplicationData.Current.LocalFolder.Path.ToString() + "/track_1.mp3";
         FileStream _stream = new FileStream(_path, FileMode.Open);
         BinaryReader _binaryReader = new BinaryReader(_stream);
         int _dataSize = _binaryReader.ReadInt32();
         byte[] _byteBuffer = _binaryReader.ReadBytes(_dataSize);
    
         int _sizeFloat = sizeof(float);
         float[] _floatBuffer = new float[_byteBuffer.Length / _sizeFloat];
         for (int i = 0, j = 0; i < _byteBuffer.Length - _sizeFloat; i += _sizeFloat, j++)
         {
             _floatBuffer[j] = BitConverter.ToSingle(_byteBuffer, i);
         }
         return _floatBuffer;
     }
    
like image 233
D. Viktor Avatar asked Feb 27 '17 10:02

D. Viktor


1 Answers

Another way to read samples from an audio file in UWP is using AudioGraph API. It will work for all audio formats that Windows10 supports

Here is a sample code

namespace AudioGraphAPI_read_samples_from_file
{
    // App opens a file using FileOpenPicker and reads samples into array of 
    // floats using AudioGragh API
// Declare COM interface to access AudioBuffer
[ComImport]
[Guid("5B0D3235-4DBA-4D44-865E-8F1D0E4FD04D")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
unsafe interface IMemoryBufferByteAccess
{
    void GetBuffer(out byte* buffer, out uint capacity);
}

public sealed partial class MainPage : Page
{
    StorageFile mediaFile;

    AudioGraph audioGraph;
    AudioFileInputNode fileInputNode;
    AudioFrameOutputNode frameOutputNode;

    /// <summary>
    /// We are going to fill this array with audio samples
    /// This app loads only one channel 
    /// </summary>
    float[] audioData;
    /// <summary>
    /// Current position in audioData array for loading audio samples 
    /// </summary>
    int audioDataCurrentPosition = 0;

    public MainPage()
    {
        this.InitializeComponent();            
    }

    private async void Open_Button_Click(object sender, RoutedEventArgs e)
    {
        // We ask user to pick an audio file
        FileOpenPicker filePicker = new FileOpenPicker();
        filePicker.SuggestedStartLocation = PickerLocationId.MusicLibrary;
        filePicker.FileTypeFilter.Add(".mp3");
        filePicker.FileTypeFilter.Add(".wav");
        filePicker.FileTypeFilter.Add(".wma");
        filePicker.FileTypeFilter.Add(".m4a");
        filePicker.ViewMode = PickerViewMode.Thumbnail;
        mediaFile = await filePicker.PickSingleFileAsync();

        if (mediaFile == null)
        {
            return;
        }

        // We load samples from file
        await LoadAudioFromFile(mediaFile);

        // We wait 5 sec
        await Task.Delay(5000);

        if (audioData == null)
        {
            ShowMessage("Error loading samples");
            return;
        }

        // After LoadAudioFromFile method finished we can use audioData
        // For example we can find max amplitude
        float max = audioData[0];
        for (int i = 1; i < audioData.Length; i++)
            if (Math.Abs(audioData[i]) > Math.Abs(max))
                max = audioData[i];
        ShowMessage("Maximum is " + max.ToString());
    }

    private async void ShowMessage(string Message)
    {
        var dialog = new MessageDialog(Message);
        await dialog.ShowAsync();
    }

    private async Task LoadAudioFromFile(StorageFile file)
    {
        // We initialize an instance of AudioGraph
        AudioGraphSettings settings = 
            new AudioGraphSettings(
                Windows.Media.Render.AudioRenderCategory.Media
                );
        CreateAudioGraphResult result1 = await AudioGraph.CreateAsync(settings);
        if (result1.Status != AudioGraphCreationStatus.Success)
        {
            ShowMessage("AudioGraph creation error: " + result1.Status.ToString());
        }
        audioGraph = result1.Graph;

        if (audioGraph == null)
            return;

        // We initialize FileInputNode
        CreateAudioFileInputNodeResult result2 = 
            await audioGraph.CreateFileInputNodeAsync(file);
        if (result2.Status != AudioFileNodeCreationStatus.Success)
        {
            ShowMessage("FileInputNode creation error: " + result2.Status.ToString());
        }
        fileInputNode = result2.FileInputNode;

        if (fileInputNode == null)
            return;

        // We read audio file encoding properties to pass them to FrameOutputNode creator
        AudioEncodingProperties audioEncodingProperties = fileInputNode.EncodingProperties;

        // We initialize FrameOutputNode and connect it to fileInputNode
        frameOutputNode = audioGraph.CreateFrameOutputNode(audioEncodingProperties);
        fileInputNode.AddOutgoingConnection(frameOutputNode);

        // We add a handler achiving the end of a file
        fileInputNode.FileCompleted += FileInput_FileCompleted;
        // We add a handler which will transfer every audio frame into audioData 
        audioGraph.QuantumStarted += AudioGraph_QuantumStarted;

        // We initialize audioData
        int numOfSamples = (int)Math.Ceiling(
            (decimal)0.0000001
            * fileInputNode.Duration.Ticks
            * fileInputNode.EncodingProperties.SampleRate
            );
        audioData = new float[numOfSamples];

        audioDataCurrentPosition = 0;

        // We start process which will read audio file frame by frame
        // and will generated events QuantumStarted when a frame is in memory
        audioGraph.Start();

    }

    private void FileInput_FileCompleted(AudioFileInputNode sender, object args)
    {
        audioGraph.Stop();
    }

    private void AudioGraph_QuantumStarted(AudioGraph sender, object args)
    {
        AudioFrame frame = frameOutputNode.GetFrame();
        ProcessInputFrame(frame);

    }

    unsafe private void ProcessInputFrame(AudioFrame frame)
    {
        using (AudioBuffer buffer = frame.LockBuffer(AudioBufferAccessMode.Read))
        using (IMemoryBufferReference reference = buffer.CreateReference())
        {
            // We get data from current buffer
            ((IMemoryBufferByteAccess)reference).GetBuffer(
                out byte* dataInBytes,
                out uint capacityInBytes
                );
            // We discard first frame; it's full of zeros because of latency
            if (audioGraph.CompletedQuantumCount == 1) return;

            float* dataInFloat = (float*)dataInBytes;
            uint capacityInFloat = capacityInBytes / sizeof(float);
            // Number of channels defines step between samples in buffer
            uint step = fileInputNode.EncodingProperties.ChannelCount;
            // We transfer audio samples from buffer into audioData
            for (uint i = 0; i < capacityInFloat; i += step)
            {
                if (audioDataCurrentPosition < audioData.Length)
                {
                    audioData[audioDataCurrentPosition] = dataInFloat[i];
                    audioDataCurrentPosition++;
                }
            }
        }
    }
}

}

Edited: It solves the problem because it reads samples from a file into a float array

like image 190
Dmitry Kh Avatar answered Sep 19 '22 00:09

Dmitry Kh