Can't get image::load_from_memory() to work when compiled to WebAssembly

Question

I am trying to load an image from JavaScript to WebAssembly with Rust using the image crate.

I have the following Rust code:

extern crate image;
extern crate libc;

use libc::c_void;
use std::mem;

#[no_mangle]
pub extern "C" fn alloc(size: usize) -> *mut c_void {
    let mut buf = Vec::with_capacity(size);
    let ptr = buf.as_mut_ptr();
    mem::forget(buf);

    return ptr as *mut c_void;
}

#[no_mangle]
pub extern "C" fn read_img(buff_ptr: *mut u8, buff_len: usize) -> *mut i32 {
    let mut img: Vec<u8> = unsafe { Vec::from_raw_parts(buff_ptr, buff_len, buff_len) };
    let ok = Box::new([333]);
    let err = Box::new([331]);

    return match image::load_from_memory(&img) {
        Ok(img) => Box::into_raw(ok) as *mut i32,
        Err(_) => Box::into_raw(err) as *mut i32,
    };
}

fn main() {}

which I compile using the following tools:

cargo +nightly build --target wasm32-unknown-unknown --release

In the read_img() function, I naively handle errors via two vectors: [333] for OK and [331] for any error. I read these vectors on the JavaScript side to know if the image was loaded successfully.

The load_from_memory method fails because I get the [331] vector. If I replace the load_from_memory method with the guess_format method, I get the [333] vector. I also did some pattern matching for PNG and JPG and it reads the buffer correctly.

I couldn't find how can I more thoroughly debug such behaviour.

On the JavaScript part, I simply load up the image's arrayBuffer into WASM's shared memory.

Here is what I am doing on the JavaScript side:

function compile(wasmFile = 'distil_wasm.gc.wasm') {
    return fetch(wasmFile)
        .then(r => r.arrayBuffer())
        .then(r => {
            let module = new WebAssembly.Module(r);
            let importObject = {}
            for (let imp of WebAssembly.Module.imports(module)) {
                if (typeof importObject[imp.module] === "undefined")
                    importObject[imp.module] = {};
                switch (imp.kind) {
                case "function": importObject[imp.module][imp.name] = () => {}; break;
                case "table": importObject[imp.module][imp.name] = new WebAssembly.Table({ initial: 256, maximum: 256, element: "anyfunc" }); break;
                case "memory": importObject[imp.module][imp.name] = new WebAssembly.Memory({ initial: 256 }); break;
                case "global": importObject[imp.module][imp.name] = 0; break;
                }
            }

            return WebAssembly.instantiate(r, importObject);
        });
}

function loadImgIntoMem(img, memory, alloc) {
    return new Promise(resolve => {
        fetch(img)
            .then(r => r.arrayBuffer())
            .then(buff => {
                const imgPtr = alloc(buff.byteLength);
                const imgHeap = new Uint8Array(memory.buffer, imgPtr, buff.byteLength);

                imgHeap.set(new Uint8Array(buff));

                resolve({ imgPtr, len: buff.byteLength });
            });
    });
}


function run(img) {
    return compile().then(m => {
        return loadImgIntoMem(img, m.instance.exports.memory, m.instance.exports.alloc).then(r => {
            window.WASM = m;
            return m.instance.exports.read_img(r.imgPtr, r.len);
        });
    });
}

run('img-2.jpg')
   .then(ptr => console.log(new Int32Array(WASM.instance.exports.memory.buffer, ptr, 1)))

This console logs:

Int32Array [ 331 ]

Answer 1

It's basically impossible to debug things without access to a debugger or the ability to print out messages. Because of this, I ported your code to use wasm-bindgen, purely for the ability to access the console from inside Rust code:

#![feature(proc_macro, wasm_custom_section, wasm_import_module)]

extern crate wasm_bindgen;
extern crate image;

use wasm_bindgen::prelude::*;
use std::mem;

pub mod console {
    use wasm_bindgen::prelude::*;

    #[wasm_bindgen]
    extern {
        #[wasm_bindgen(js_namespace = console)]
        pub fn log(s: &str);
    }
}

#[wasm_bindgen]
pub fn alloc(len: usize) -> *mut u8 {
    let mut buf = Vec::with_capacity(len);
    let ptr = buf.as_mut_ptr();
    mem::forget(buf);
    ptr
}

#[wasm_bindgen]
pub fn read_img(ptr: *mut u8, len: usize) {
    let img = unsafe { Vec::from_raw_parts(ptr, len, len) };

    if let Err(e) = image::load_from_memory(&img) {
        console::log(&e.to_string());
    }
}

The updated JavaScript:

const js = import("./imaj_bg");

async function loadImgIntoMem(img, { alloc, memory }) {
  const resp = await fetch(img);
  const buf = await resp.arrayBuffer();

  const len = buf.byteLength;
  const ptr = alloc(len);

  const imgArray = new Uint8Array(memory.buffer, ptr, len);    
  imgArray.set(new Uint8Array(buf));

  return { ptr, len };
}

async function go(js) {
  const { ptr, len } = await loadImgIntoMem('cat.jpg', js);
  js.read_img(ptr, len);
};

js.then(go);

Building and serving the code:

$ cargo build --target wasm32-unknown-unknown --release
$ wasm-bindgen target/wasm32-unknown-unknown/release/imaj.wasm --out-dir=.
$ yarn serve

Accessing the page and reviewing the console log shows this anticlimactic message:

operation not supported on wasm yet

The truth is that there's large parts of the Rust standard library that don't exist yet in WebAssembly. Many of these are stubbed out to return this error.

I don't know exactly which platform support that is missing for your code. The most obvious one is threading, required by the jpeg_rayon and hdr feature, but turning off all of image's features except jpeg still reports same error. It's likely there's something else needed.

However, it does seem to be specific to a given image codec. If you try the same code but load a PNG image, it's successful:

pub fn read_img(ptr: *mut u8, len: usize) {
    let img = unsafe { Vec::from_raw_parts(ptr, len, len) };

    let img = match image::load_from_memory(&img) {
        Ok(i) => i,
        Err(e) => {
            console::log(&e.to_string());
            return;
        }
    };

    console::log(&format!("{:?}", img.to_rgba()));
}

ImageBuffer { width: 305, height: 314, _phantom: PhantomData, data: [255, 255, 255, 0 /* remaining pixels skipped */

This indicates that the JPEG code does not yet work with WASM. A given codec may or may not work yet; it's probably best to file issues with the upstream maintainers.