Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to access an object from the topology context into a bolt when using storm?

Tags:

apache-storm

We need to pass an object when creating a topology so that the bolt can access that and do some further processing based on that object. Is it possible to pass the object via TopplogyContext and if yes, how? Or are there any other ways to pass an object when submitting a topology, before submitting so that bolt can have a handle/control on it?

We need to pass the object via a context so that all bolts can access it and there is no need to force an implementation of constructor in all the bolts for that topology. So, Wanted to know if any API exists to do the same?

like image 851
user1996684 Avatar asked Aug 02 '13 05:08

user1996684


Video Answer


2 Answers

You can pass the object in the storm configuration map, provided that it is serializable. In the prepare() method of any spout or bolt in the topology you can retrieve this object.

This is how you put your object in the configuration map on topology submission:

Config conf = new Config();
MyObject myPreciousObject = new MyObject("precious");
conf.put("my.object",myPreciousObject);

StormSubmitter.submitTopology(args[0], conf, builder.createTopology());

This is how you retrieve it in the prepare() method of a bolt or spout:

prepare(Map stormConf,TopologyContext context) {

   MyObject myPreciousObject = (MyObject) stormConf.get("my.object");

} 
like image 108
ChrisBlom Avatar answered Oct 19 '22 11:10

ChrisBlom


I am not very sure what you mean, but your bolt class can always take a parameter on initialization, and you can initialize that with the object you want to pass when creating the topology.

TopologyBuilder builder = new TopologyBuilder();
builder.setSpout( "spout", new mySpout() );
builder.setBolt( "bolt", new myBolt1(myObj) ).shuffleGrouping("spout");

And your bolt constructor could take this object as the argument.

Edit: If you want the data to be accessible without passing it explicitly to a constructor, you can again always make a static class to store this data and access it from the bolt objects

like image 3
Munim Avatar answered Oct 19 '22 11:10

Munim