I am looking at possible technology choices for queues (or perhaps streams are a better description) in a JVM-based system.
Some requirements:
- Must be accessible from the JVM / Java.
- Queues must support sizes larger than the JVM heap, possibly bigger than all available RAM. Thus, support for utilizing the disk (or network) for storage is implied.
- Queues do not currently need to be durable past the process lifetime.
- Most uses of the queue will have a single producer and a single consumer. Concurrency for any particular queue is thus not an issue. (Obviously concurrency is important across queues.)
- Queues are ad-hoc and temporary. They pop into existence, are filled, are drained, and go away.
- Small queues should preferably stay in memory, then shift to slower storage based on resource availability. This requirement could be met above the queuing technology.
I am examining several options but am curious what options I am missing?
Use one of available JMS implementations. For example ActiveMQ or Qpid from Jakarta.