R: how to serialize C/C++ data that lives in the heap?

Question

Suppose I want to wrap some C or C++ code which contains arrays or vectors that cannot be automatically mapped to R types by Rcpp, but which I need to pass to C/C++ functions that would output valid R objects. For example:

typedef union {
    size_t val_sizet;
    long double val_longdbl;
    } weird_struct
std::vector<weird_struct> an_unmappable_obj;
an_unmappable_obj.resize(2);
an_unmappable_obj[0].val_sizet = 1e20;
an_unmappable_obj[1].val_longdbl = 1.5 * 1e20;

Since this is a vector of a type that cannot be converted to any of R’s native types, I’m wondering how can I return and handle these objects inside R/Rcpp in such a way that the vector (or C array containing the same values) could be serialzed through saveRDS and its values restored after readRDS.

I guess one way of doing it would be through memcpy'ing the contents of the object to some C++ vector of a type that could be converted to Rcpp's 'NumericVector` or similar, then force-casting its first element to an array of the desired C types when it needs to be used, but I'm wondering if there's a better solution.

Answer 1

If you just want to save C++ data for later in the same session, the easiest way is to use external pointers. E.g.:

// [[Rcpp::export]]
Rcpp::XPtr< std::vector<double> > xptr_example() {
  std::vector<double> * x = new std::vector<double>(10);
  Rcpp::XPtr< std::vector<double> > p(x, true);
  return p;
}

If instead you still just want to serialize, there are lots of options but you will have to write some additional custom code.

As you say, you could cast and memcpy into a R vector (use RawVector instead of NumericVector), but you have to be careful that your class has only "plain old data" and nothing special like pointers or file handlers.

More formal serialization options and examples using Rcpp can be seen here and here.

Answer 2

I was going to suggest using the cereal library together with Rcereal, but that seems to be difficult to use in conjunction with a union. If one uses the more C++-like boost::variant, that works quite nicely, though. The idea is to serialize to object to a raw vector, which could then be saved and restored with saveRDS and readRDS. Here some sample code:

#include <Rcpp.h>
// [[Rcpp::plugins("cpp11")]]
// [[Rcpp::depends(BH, Rcereal)]]
#include <boost/variant.hpp>
#include <cereal/types/vector.hpp>
#include <cereal/types/boost_variant.hpp>
#include <cereal/archives/binary.hpp>
#include <sstream>

// [[Rcpp::export]]
Rcpp::RawVector get_weird_vec() {
  std::vector<boost::variant<std::size_t, long double>> an_unmappable_obj;
  an_unmappable_obj.push_back((std::size_t) 3e9);
  an_unmappable_obj.push_back((long double) 1.5 * 1e20);

  std::ostringstream os;
  cereal::BinaryOutputArchive archive(os);
  archive(an_unmappable_obj);

  std::string out = os.str();

  Rcpp::RawVector res(out.size());
  std::copy(out.begin(), out.end(), res.begin());
  return res;
}

// [[Rcpp::export]]
void process_weird_vec(Rcpp::RawVector src) {
  std::stringstream ss;
  ss.write(reinterpret_cast<char*>(&src[0]), src.size());
  cereal::BinaryInputArchive archive(ss); 

  std::vector<boost::variant<std::size_t, long double>> an_unmappable_obj;
  archive(an_unmappable_obj);

  Rcpp::Rcout << an_unmappable_obj[0] << std::endl;
  Rcpp::Rcout << an_unmappable_obj[1] << std::endl;
}

/*** R
raw <- get_weird_vec()
raw
process_weird_vec(raw)
*/

Output:

> raw <- get_weird_vec()

> raw
 [1] 02 00 00 00 00 00 00 00 00 00 00 00 00 5e d0 b2 01 00 00 00 00 80 49 41 d4
[26] b0 1a 82 42 40 08 02

> process_weird_vec(raw)
3000000000
1.5e+20

References:

• Cereal documentation
• Rcpp gallery
• Keep XPtr for multiple sessions
• https://lists.r-forge.r-project.org/pipermail/rcpp-devel/2010-October/001179.html
• https://github.com/bearloga/learning-rcpp#serialization