Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Sent different tuples from 1 spout to different bolt in Apache Storm

Is it possible to sent different tuples from 1 spout to different bolt in Apache Storm? For instance, I had Spout A, which need to sent out Tuple B to Bolt C and Tuple D to Bolt E. How should I implement it using spout in Java? I mean how to write the code.

OutputCollector.emit(new Values(B, C))?
like image 542
Toshihiko Avatar asked May 21 '15 23:05

Toshihiko


People also ask

What is spout in Apache Storm?

Spouts. A spout is a source of streams in a topology. Generally spouts will read tuples from an external source and emit them into the topology (e.g. a Kestrel queue or the Twitter API). Spouts can either be reliable or unreliable.

Which of the following method is used for performing transformation in Bolt Apache Storm?

The main method in bolts is the execute method which takes in as input a new tuple. Bolts emit new tuples using the OutputCollector object.

Which of the following method of Storm topology is used to initialize the spout?

open − Provides the spout with an environment to execute. The executors will run this method to initialize the spout.

What is tuple in Apache Storm?

The tuple is the main data structure in Storm. A tuple is a named list of values, where each value can be any type. Tuples are dynamically typed – the types of the fields do not need to be declared. Tuples have helper methods like getInteger and getString to get field values without having to cast the result.


1 Answers

To emit tuples to different bolts from one Spout you can use named streams as follows :

Spout

@Override
public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
    outputFieldsDeclarer.declareStream("streamA", new Fields("A"));
    outputFieldsDeclarer.declareStream("streamB", new Fields("B"));
}


@Override
public void nextTuple() {
    outputCollector.emit("streamA", new Values("A"));
    outputCollector.emit("streamB", new Values("B"));
}

Then, each bolt subscribes to a specific stream :

builder.setBolt("MyBoltA", new BoltA()).shuffleGrouping("MySpout", "streamA"); 
builder.setBolt("MyBoltB", new BoltB()).shuffleGrouping("MySpout", "streamB");

Finally, if a bolt subscribes to several streams, you can use the following method to know from which stream a tuple has been emitted :

tuple.getSourceStreamId()
like image 193
fhussonnois Avatar answered Sep 20 '22 14:09

fhussonnois