Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reading large user provided file from emscripten, chunk at a time

I am looking for an API that let me read file provided via (or any other method that allows to access users files from browser) chunk at a time. I'm reading large files, so I don't want to load entire file to memory.

My use case is, I'm using ffmpeg library compiled with emcc and I wan't to process multimedia files by it. I could implement my own AVIOContext but to do that, I would need equivalent of C functions fread and fseek.

I was looking at FS API with WORKERFS file system type, but it is not clear to me that I could mount it from worker thread with File object from DOM.

like image 877
sc0ty Avatar asked Sep 21 '25 03:09

sc0ty


1 Answers

I was able to mount file with WORKERFS from worker thread.

Minimal example:

main.html:

<html>
<head>
<script>
    const worker = new Worker("worker.js");
    function onClick() {
        const f = document.getElementById("in-file").files[0];
        worker.postMessage([ f ]);
    }
</script>
</head>
<body>
    <input type="file" id="in-file" />
    <input type="button" onClick="onClick()" value="ok" />
</body>
</html>

worker.js

onmessage = function(e) {
    const f = e.data[0];

    FS.mkdir('/work');
    FS.mount(WORKERFS, { files: [f] }, '/work');

    console.log(Module.read_file('/work/' + f.name));
}

self.importScripts('hello.js');

hello.js is compiled hello.cpp with (emcc --bind -lworkerfs.js -o hello.js hello.cpp -s WASM=1):

#include <cstdio>
#include <string>
#include <iostream>
#include <fstream>
#include <emscripten/bind.h>
using namespace emscripten;

std::string read_file(const std::string &fn)
{
    std::ifstream f(fn);
    std::string line;
    std::getline(f, line);
    return line;
}

EMSCRIPTEN_BINDINGS(hello) {
    function("read_file", &read_file);
}
like image 117
sc0ty Avatar answered Sep 23 '25 06:09

sc0ty



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!