Asynchronous event in Spring taking lot of time to execute(get its turn)

Question

I have an application where I need to trigger email whenever a REST call is made for an endpoint. The design is that whenever a REST call is invoked, I save the data in the db, emit an asynchrnous event and return.

My problem is that due to huge no of requests that keep coming, the async events which are emitted do not get a chance for quite lot of time. Sometimes as the server is up for some weeks, the delay keeps increasing.

The scenario

1. Server endpoint is invoked
2. Server saves data to db, emits a Spring Asynchronous event
3. Returns from the endpoint

The call 2 is getting delayed as the listener is invoked quite late sometimes.

public class DataController {   
    @Inject
    ApplicationEventPublisher eventPublisher;

    @RequestMapping(value = "data", method = RequestMethod.POST)
    @ResponseStatus(HttpStatus.NO_CONTENT)
    public void addData(@RequestBody DataDTO data) {
        dataService.addData(data);
        eventPublisher.publishEvent(new DataRequest(new DataDTO());
    }
}



public class DataRequest extends ApplicationEvent {


    private DataDTO dataDTO;

    public DataRequest(DataDTO dataDTO) {
        super(dataDTO);
        this.dataDTO = dataDTO;
    }
}

@Component
public class DataListener {

    @EventListener
    @Async
    private void dataListener(DataDTO dataDTO) {
        // Send email
    }
}

Since it is an Async event , the JVM gives the dataListener chance to execute very late. Sometimes the events triggered earlier gets chance late than the ones that were triggered after that.

So 2 fundamental problems

1. Emails delayed. Delay can range from 1 min, to 4 hours to 8 days etc
2. If an event is triggered at 12 PM to send email to [email protected], and another at 12:15 PM which sends email to [email protected], then there are chances of [email protected] receiving email before [email protected].

Appreciate your help

Answer 1

Spring Asynchronous event is limited to the size of the Thread pool and as soon as the incoming requests are higher than the size of active threads there will be delays.

You need to use a Message Queue like RabbitMQ, Kafka, etc. Your architecture should be changed to do the following;

1. Serialize a JSON message in the REST Endpoint with all information like to email address, database entry data, etc and just store that JSON message in the message queue and return a status code
2. There must be consumers for message queue (separate Java applications) which poll or get notified when there is data in the message queue.
3. These consumers should de-serialize the JSON message, save an entry in the database and send an email.

With this architecture you can increase consumers at times of high load and thus scale as required.