Convert MapWritable to string

Question

I am using MapWritable in my Hadoop Map Reduce program. When I emit the MapWritable map as new Text (mapName.toString()), I get the following output

key1      org.apache.hadoop.io.MapWritable@396cbd97 
key2      org.apache.hadoop.io.MapWritable@17991de1 
key3      org.apache.hadoop.io.MapWritable@18f63055 

Can you please let me know how to print this in a simple way instead of iterating over it ? I know we can print a HashMap simply by calling mapName.toString().

I'm using this in a Stripes approach and I see very bad performance compared to Pairs approach (Ignore this line if you are not aware of Design Patterns in Map-Reduce. I added this statement to give more information to those who know it).

Any pointers will be appreciated.

Answer 1

The only way to accomplish this would be to extend MapWritable and override the toString to output what you desire (this could involve either iterating over it when toString is called or when the map is created/modified and storing the string as a field depending on your needs).

MapWritable's javadoc show that it inherits its toString method from Object - the reason you see more useful output for HashMap is because it is inheriting AbstractMap's toString method, which overrides Object's, from the javadoc:

Returns a string representation of this map. The string representation consists of a list of key-value mappings in the order returned by the map's entrySet view's iterator, enclosed in braces ("{}"). Adjacent mappings are separated by the characters ", " (comma and space). Each key-value mapping is rendered as the key followed by an equals sign ("=") followed by the associated value. Keys and values are converted to strings as by String.valueOf(Object).

Answer 2

Since MapWritable is implemented in terms of a Map, it is trivial to get this working, but the member is private. You could implement it yourself if you were willing to use reflection.

See https://issues.apache.org/jira/browse/HADOOP-6842 for a bug report and potential workaround. A patch has been submitted to Hadoop to delegate the toString method properly, but unfortunately, it hasn't been applied and doesn't seem to have much traction. You could also apply the patch directly to your own copy of Hadoop, but that is not ideal either. I would recommend voting for the bug and maybe commenting on the issue so that the Hadoop maintainers realize this is impacting users.

UPDATE: This bug has now been fixed in Hadoop 2.8.0.