How to simply set up a serializer for storm Bolt/Spout members?

Question

I have a bolt that uses a factory interface to create the tools it uses. It creates those tools when prepare is called. The Factory implementation only has basic members (strings, integers) that should be serializable by default.

When I run my topology I get a NotSerializableException, coming from the factory implementation. I was wondering how I could register a serializer for the factory.

here's an example:

public class Demo extends BaseRichBolt {

    public static interface IExecutor
    {
        public void execute( Tuple tuple );
    }

    public static interface IExecutorCreator
    {
        public IExecutor create( Map map, TopologyContext tc, OutputCollector oc  );
    };

    public static class DummyExecutor implements IExecutor
    {
        public void execute( Tuple tuple ) {}
    };

    public static class DummyExecutorCreator implements IExecutorCreator
    {
        String name_;
        public DummyExecutorCreator(String name) { this.name_ = name;}
        public IExecutor create(Map map, TopologyContext tc, OutputCollector oc) {
            return new DummyExecutor();
        }
    };

    public void declareOutputFields(OutputFieldsDeclarer ofd) {

    }

    private IExecutor           executor_;
    private IExecutorCreator    creator_;

    public Demo(IExecutorCreator creator)
    {
        this.creator_ = creator;
    }

    public void prepare(Map map, TopologyContext tc, OutputCollector oc) {
        this.executor_ = this.creator_.create(map, tc, oc);
    }

    public void execute(Tuple tuple) {
        this.executor_.execute(tuple);
    }

}

And I get this error when I try to run it in a topology: java.io.NotSerializableException: Demo$DummyExecutorCreator

As a side note I'm starting to wonder, why don't Storm have you to register factories instead of Bolts and Spouts. since in the end they get serialized and copied across the different threads, it'd be probably better to just give storm a mean to generate those bolts and separate the concerns.

Answer 1

Don't set any values in your constructor. The constructor is called when the topology is first built by the driver and before the topology is submitted to the cluster. That's why any instance variables you set, whether by class initialization or in the constructor, must be serializable.

Instead, simply leave those instance variables null and set them in the prepare() method. When you do that you won't need to serialize any values, so this works for non-serializable instance variables, too.