How to prioritize a certain request in java?

Question

I feel like this is such a niche question so I will try to explain it as best possible as I can.

Intro: I'm sending ~500 requests / second and I feel like the more requests I send the slower the requests are handled (it becomes noticeably slower at some point)

Question: So the question is in Java is there any way to prioritize a request? Any solution that I am seeking is to optimize the speed of such request.. So any answer that would take time before the request is sent is not of my concern.

INFO: (I hope this is sufficient if not please tell me!)

• The library I am using is apache httpclients (however I can switch if the solutions calls for it)
• I also am multi threading the requests on one server/pc. I hope this is helpful information.
• CPU Usage varies from (5-15%) - I believe these are the measurements

I am sending 2 types of request and I only need to prioritize 1 type

1. HTTP GET Request - HTML Response expected
2. HTTP POST Request - JSON response expected (although I do not need the response)

#2 is the request that I want to prioritize. I send this request very little but when I send it I need it to be as quick as possible.

Solutions thought of: The only solution I have come up with is to stop/end all of the live connections in order to execute the request I want, however I think that doing so will take a considerable amount of time causing the solution to become a waste of time.

Note: You could say I am an idiot in this area so if the solution is non existent or obvious I am sorry, also if there is a duplicate I am also sorry.. I could not find any questions even close to this.

Answer 1

This may be a workaround, as it must be executed before the requests are sent. Taking into account your use case (500 requests at sec), my suggestion is to send first the most critical ones, by using a PriorityQueue.

As you already batch the messages in order to send them, this approach would help into ordering the batched messages in base of the set priority.

---

You could first wrap the requests into another entity that holds a priority field. For example, an skeleton/base PriorityRequest class:

public class PriorityRequest implements Comparable<PriorityRequest> 
{
    public int priority;
    public PriorityRequest(int priority) 
    {
       this.priority=priority;
    }

    @Override
    public int compareTo(PriorityRequest request) 
    {
       return Integer.compare(request.priority,this.priority);
    }
}

---

And declare both children, HttpPost and HttpGet:

public class PriorityHttpPost extends PriorityRequest 
{
    public HttpPost post;
    public PriorityHttpPost(int priority, HttpPost post) 
    {
       super(priority);
       this.post=post;
    }
}        

---

public class PriorityHttpGet extends PriorityRequest 
{
    public HttpGet get;
    public PriorityHttpGet(int priority, HttpGet get) 
    {
       super(priority);
       this.get=get;
    }
}

---

So, while you create the requests, you could insert them into the queue so they get automatically located in base of their priority:

Queue<PriorityRequest> requestQueue = new PriorityQueue<>();

/*into the batch mechanism*/
requestQueue.add(new PriorityHttpPost(6,httpPost));
//...
requestQueue.add(new PriorityHttpGet(99,httpGet));
//...

---

This way, you guarantee the requests with higher priority to leave the queue before the lesser priority ones, as they will be ordered in descending order.

Queue- | Get  (99) | --> out
       | Get  (9)  |
       | Post (6)  |
       | Get  (3)  |
       | Post (1)  |

Queue- | Get  (9)  | --> out
       | Post (6)  |  
       | Get  (3)  |
       | Post (1)  |

        (...)

---

Just to finish, a little extra feature of this approach (in certain use cases) would consist of being able to define which elements go first and which go last:

requestQueue.add(new PriorityHttpPost(INTEGER.MAX_VALUE, httpPostMax));
requestQueue.add(new PriorityHttpPost(INTEGER.MAX_VALUE-1, httpPostVery));
requestQueue.add(new PriorityHttpPost(INTEGER.MIN_VALUE+1, httpPostNotVery));
requestQueue.add(new PriorityHttpPost(INTEGER.MIN_VALUE, httpPostNoOneCares));

--

perfect world, yeah, i know..

Queue- | Post (MAX)   | --> out
       | Post (MAX-1) |
       | ............ |
       | ............ |
       | Post (MIN+1) |
       | Post (MIN)   |