Menu
TL;DR; I need to know if there's a lib with persistent blocking queue that performatic.
I hava a classic producer/consumers program. They share a LinkedBlockingQueue to share the data, and I use BlockingQueue#take method in the Consumers, as I need them to live forever waiting for new elements.
The problem is I have LOTS of data and I can't lose them. Even after the consumers stops, the producer can persist to generate some data. I am thinking about implementing my BlockingQueue ta uses H2 behind to store/get the data after some threshold is reached. My main problem is that I need performance and I need to consume the elements in the order they are created.
Is there an implementation of persistent blocking queue that I can use for something like this? If it doesn't, any sugestions for me to achieve something like this?
468
BlockingQueue is a java Queue that support operations that wait for the queue to become non-empty when retrieving and removing an element, and wait for space to become available in the queue when adding an element.
BlockingQueue implementations are thread-safe. All queuing methods achieve their effects atomically using internal locks or other forms of concurrency control.
Blocking vs Non-Blocking QueueThe producers will wait for available capacity before adding elements, while consumers will wait until the queue is empty. In those cases, the non-blocking queue will either throw an exception or return a special value, like null or false.
Here we have a blockingQueue that has a capacity equal to 10. It means that when a producer tries to add an element to an already full queue, depending on a method that was used to add it (offer(), add() or put()), it will block until space for inserting object becomes available. Otherwise, the operations will fail.
I would use ActiveMQ lib and Spring JMS, here is a usage example
start broker
BrokerService broker = new BrokerService();
broker.addConnector("tcp://localhost:61616");
broker.start();
read msg
ConnectionFactory cf = new ActiveMQConnectionFactory("tcp://localhost:61616");
JmsTemplate t = new JmsTemplate(cf);
Message msg = t.receive();
send message
ConnectionFactory cf = new ActiveMQConnectionFactory("tcp://localhost:61616");
JmsTemplate t = new JmsTemplate(cf);
t.send("test", new MessageCreator() {
public Message createMessage(Session session) throws JMSException {
return session.createTextMessage("test");
}
});
You can try ActiveMQ. The ActiveMQ can write to your file system so if the producer is generating many more elements than the consumer can take you either have a lot of blocking (on whatever the upper bound of the queue is) or excessive data (if there is no upper bound to the queue).
37
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
