nodejs native c++ npm module memory error, cairo image processing

Question

I've been bugging TJ on node-canvas about a code speed up I'm working on in a fork of a node module he authored and maintains.

I found Canvas.toBuffer() to be killing our pipeline resources and created an alternative that would simply convert from Canvas into an Image without going through a png buffer/media url. The problem is that cairo is a mysterious beast, and there's an additional level of concern about memory allocated within node modules as not to get GC'd by mother v8. I've added the proper HandleScopes to all required functions which access V8 data.

I was able to test the Canvas.loadImage(image) method thousands of times on my mac setup (6.18), as well as stand alone tests on our ubuntu/production servers running the same version of node. But when the code is run as a background process/server and coordinated by Gearman I'm getting some "interesting" memory/segfaults.

In addition I'm having trouble calling any of the methods of classes defined in node-canvas that aren't inline within header files. As a side question What's the best way to create common native source code packages that other node modules can rely on?

I've tried recreating the problem and running it with gdb, node_g, and all the node modules built with symbols and debug flags. But the error crops up in a lib outside of the source I can get a stack trace for.

for reference here's where I call loadImageData and while it runs locally under a variety of conditions, in our production environment when carefully tucked away within a frame server it appears to be causing segfaults (spent the day yesterday trying to gdb node_g our server code but the frame servers are kicked off by gearman... TL;DR didn't get a root cause stack trace)

https://github.com/victusfate/node-canvas/blob/master/src/Canvas.cc#L497

Handle<Value>
 Canvas::LoadImage(const Arguments &args) {
   HandleScope scope;
   LogStream mout(LOG_DEBUG,"node-canvas.paint.ccode.Canvas.LoadImage");    
   mout << "Canvas::LoadImage top " << LogStream::endl;

   Canvas *canvas = ObjectWrap::Unwrap<Canvas>(args.This());
   if (args.Length() < 1) {
     mout << "Canvas::LoadImage Error requires one argument of Image type " << LogStream::endl;
     return ThrowException(Exception::TypeError(String::New("Canvas::LoadImage requires one argument of Image type")));
   }

   Local<Object> obj = args[0]->ToObject();
   Image *img = ObjectWrap::Unwrap<Image>(obj);
   canvas->loadImageData(img);
   return Undefined();
}  

void Canvas::loadImageData(Image *img) {
  LogStream mout(LOG_DEBUG,"node-canvas.paint.ccode.Canvas.loadImageData");    
  if (this->isPDF()) {
    mout << "Canvas::loadImageData pdf canvas type " << LogStream::endl;
    cairo_surface_finish(this->surface());
    closure_t *closure = (closure_t *) this->closure();

    int w = cairo_image_surface_get_width(this->surface());
    int h = cairo_image_surface_get_height(this->surface());

    img->loadFromDataBuffer(closure->data,w,h);
    mout << "Canvas::loadImageData pdf type, finished loading image" << LogStream::endl;
  }
  else {
    mout << "Canvas::loadImageData data canvas type " << LogStream::endl;
    cairo_surface_flush(this->surface());
    int w = cairo_image_surface_get_width(this->surface());
    int h = cairo_image_surface_get_height(this->surface());

    img->loadFromDataBuffer(cairo_image_surface_get_data(this->surface()),w,h);
    mout << "Canvas::loadImageData image type, finished loading image" << LogStream::endl;
  }   
}

and here's what the current method in Image looks like (I removed some commented out logging info) https://github.com/victusfate/node-canvas/blob/master/src/Image.cc#L240

/*
 * load from data buffer width*height*4 bytes
 */
cairo_status_t
Image::loadFromDataBuffer(uint8_t *buf, int width, int height) {
  this->clearData();
  int stride = cairo_format_stride_for_width (CAIRO_FORMAT_ARGB32, width); // 4*width + ?
  this->_surface = cairo_image_surface_create_for_data(buf,CAIRO_FORMAT_ARGB32,width,height,stride);
  this->data_mode = DATA_IMAGE;
  this->loaded();
  cairo_status_t status = cairo_surface_status(_surface);
  if (status) return status;
  return CAIRO_STATUS_SUCCESS;
}

Any help, pro tips, assistance, or words of encouragement would be appreciated.

Originally from google groups

Answer 1

In addition I'm having trouble calling any of the methods of classes defined in node-canvas that aren't inline within header files. As a side question What's the best way to create common native source code packages that other node modules can rely on?

While I don't have an answer for the memory issue/seg fault I was running into in our staging environment. I do have an answer for constructing reusable libraries with native node modules.

I'm using git submodules for all the independent native node modules, and added a conditional preprocessor definition for each of their wscript or binding.gyp files to specify whether or not to generate a shared object .node module.

update Alternatively unique init function names or namespaces can surround the module initialization call (moved to this setup).

In addition I'll be using this new package to aid in debugging or rewriting code sections (I can't spend too much time debugging utilization of several remote libraries).

in wscript or binding.gyp

  flags = ['-D_NAME_NODE_MODULE', '-O3', '-Wall', '-D_FILE_OFFSET_BITS=64', '-D_LARGEFILE_SOURCE', '-msse2']

then in an initialization file

#ifdef _NAME_NODE_MODULE

extern "C" {
  static void init(Handle<Object> target) {
    HandleScope scope;
    NODE_SET_METHOD(target, "someFunction", someFunction);
  }

  NODE_MODULE(moduleName, init);
}

#endif

This way a node native module is only added to when the flag is set. Otherwise it can be linked to normally (like in another node module).