STL map in Perl using SWIG

Question

This is duplicate of my question on SWIG mailing list.

I am trying to use stl containers in my SWIG bindings. Everything works perfectly except for stl map handling in Perl. On C++ side, I have

std::map<std::string, std::string> TryMap(const std::map<std::string, std::string> &map) {   std::map<std::string, std::string> modified(map);   modified["7"] = "!";   return modified; } 

SWIG config look like this

%module stl  %include "std_string.i"  %include "std_map.i" %template(StringStringMap) std::map<std::string, std::string>;  %{   #include "stl.h" %}  %include "stl.h" 

In my Python script I can call TryMap this way

print dict(stl.TryMap({'a': '4'})) 

and get beautiful output

{'a': '4', '7': '!'} 

but in Perl I call

print Dumper stl::TryMap({'a' => '4'}); 

and get an error

TypeError in method 'TryMap', argument 1 of type 'std::map< std::string,std::string > const &' at perl.pl line 7. 

I can actually do something like

my $map = stl::TryMap(stl::StringStringMap->new()); print $map->get('7'); 

and get '!', but this is not an option because there is a lot of legacy code using "TryMap" that expects normal Perl hash as its output.

I believe there is a way work this out because SWIG solves this particular problem nicely in Python and even in Perl if I use stl vectors and strings but not maps.

Is there any way to handle stl map with Perl in SWIG? I am using latest SWIG 2.0.7

UPDATE Maybe there is something wrong with perl5/std_map.i. It is too short =)

$ wc -l perl5/std_map.i python/std_map.i     74 perl5/std_map.i   305 python/std_map.i 

Answer 1

I put your C++ function into header file as an inline function for testing.

I was then able to construct a SWIG interface that does what you are looking for. It has two key parts. Firstly I wrote a typemap that will allow either a std::map, or a perl hash to be given as input to C++ functions that expect a std::map. In the case of the latter it builds a temporary map from the perl hash to use as the argument. (Which is convenient but potentially slow). The typemap picks the correct behaviour by checking what it was actually passed in.

The second part of the solution is to map some of the C++ map's member functions onto the special functions that perl uses for overloading operations on hashes. Most of these are implemented simply with %rename where the C++ function and perl functions are compatible however FIRSTKEY and NEXTKEY don't map well onto C++'s iterators, so these were implemented using %extend and (internally) another std::map to store the iteration state of the maps we're wrapping.

There are no special typemaps implemented here for returning the maps, however there is extra behaviour via the special operations that are now implemented.

The SWIG interface looks like:

%module stl  %include <std_string.i> %include <exception.i>  %rename(FETCH) std::map<std::string, std::string>::get; %rename(STORE) std::map<std::string, std::string>::set; %rename(EXISTS) std::map<std::string, std::string>::has_key; %rename(DELETE) std::map<std::string, std::string>::del; %rename(SCALAR) std::map<std::string, std::string>::size; %rename(CLEAR) std::map<std::string, std::string>::clear;  %{ #include <map> #include <string> // For iteration support, will leak if iteration stops before the end ever. static std::map<void*, std::map<std::string, std::string>::const_iterator> iterstate;  const char *current(std::map<std::string, std::string>& map) {   std::map<void*, std::map<std::string, std::string>::const_iterator>::iterator it = iterstate.find(&map);   if (it != iterstate.end() && map.end() == it->second) {     // clean up entry in the global map     iterstate.erase(it);     it = iterstate.end();   }   if (it == iterstate.end())     return NULL;   else     return it->second->first.c_str(); } %}  %extend std::map<std::string, std::string> {   std::map<std::string, std::string> *TIEHASH() {     return $self;   }   const char *FIRSTKEY() {     iterstate[$self] = $self->begin();     return current(*$self);   }   const char *NEXTKEY(const std::string&) {     ++iterstate[$self];     return current(*$self);   } }  %include <std_map.i>  %typemap(in,noblock=1) const std::map<std::string, std::string>& (void *argp=0, int res=0, $1_ltype tempmap=0) {   res = SWIG_ConvertPtr($input, &argp, $descriptor, %convertptr_flags);   if (!SWIG_IsOK(res)) {     if (SvROK($input) && SvTYPE(SvRV($input)) == SVt_PVHV) {       fprintf(stderr, "Convert HV to map\n");       tempmap = new $1_basetype;       HV *hv = (HV*)SvRV($input);       HE *hentry;       hv_iterinit(hv);       while ((hentry = hv_iternext(hv))) {         std::string *val=0;         // TODO: handle errors here         SWIG_AsPtr_std_string SWIG_PERL_CALL_ARGS_2(HeVAL(hentry), &val);         fprintf(stderr, "%s => %s\n", HeKEY(hentry), val->c_str());         (*tempmap)[HeKEY(hentry)] = *val;         delete val;       }       argp = tempmap;     }     else {       %argument_fail(res, "$type", $symname, $argnum);      }   }   if (!argp) { %argument_nullref("$type", $symname, $argnum); }   $1 = %reinterpret_cast(argp, $ltype); }  %typemap(freearg,noblock=1) const std::map<std::string, std::string>& {   delete tempmap$argnum; }  %template(StringStringMap) std::map<std::string, std::string>;  %{ #include "stl.h" %}  %include "stl.h" 

I then adapted your sample perl to test:

use Data::Dumper; use stl;  my $v = stl::TryMap(stl::StringStringMap->new()); $v->{'a'} = '1'; print Dumper $v; print Dumper stl::TryMap({'a' => '4'}); print Dumper stl::TryMap($v);   foreach my $key (keys %{$v}) {   print "$key => $v->{$key}\n"; }  print $v->{'7'}."\n"; 

Which I was able to run successfully:

Got map: 0x22bfb80 $VAR1 = bless( {                  '7' => '!',                  'a' => '1'                }, 'stl::StringStringMap' ); Convert HV to map a => 4 Got map: 0x22af710 In C++ map: a => 4 $VAR1 = bless( {                  '7' => '!',                  'a' => '4'                }, 'stl::StringStringMap' ); Got map: 0x22bfb20 In C++ map: 7 => ! In C++ map: a => 1 $VAR1 = bless( {                  '7' => '!',                  'a' => '1'                }, 'stl::StringStringMap' ); 7 => ! a => 1 ! 

You can also tie this object to a hash, for example:

use stl;  my $v = stl::TryMap(stl::StringStringMap->new()); print "$v\n";  tie %foo, "stl::StringStringMap", $v;  print $foo{'a'}."\n"; print tied(%foo)."\n"; 

In theory you can write an out typemap to set up this tie automatically on return from every function call, but so far I've not succeeded in writing a typemap that works with both the tying and the SWIG runtime type system.

It should be noted that this isn't production ready code. There's a thread safety issue for the internal map and some error handling missing too that I know of. I've also not fully tested all of hash operations work from the perl side beyond what you see above. It would also be nice to make it more generic, by interacting with the swig_map_common macro. Finally I'm not a perl guru by any means and I've not used the C API much so some caution in that area would be in order.