HandleHTTPRequest processor creating duplicate output flowfiles in nifi

Question

​Hi, I have created the handlehttprequest processor, on a standalone nifi instance which is timer driven, concurrent tasks set to 1 and run schedule is 0. I am hitting my web service with a request from postman or any browser. The output queue of the processor has one entry initially but it becomes 2 after around a minute. I have seen this issue on a single node as well as on a clustered nifi instance. Can someone give any pointers

Nifi Version: 1.7.0

Update 1: I have observed that in case the http response flow is not reached within a certain time a new http request is generated wit the same parameters. Is this expected?, if yes how can I avoid this?

Update 2: We have seen that the second flow file gets generated after exactly 60 seconds. Not sure which configuration sets this.

Update 3: I just saw that the StandardHttpContextMap that we use has a property: Request Expiration which is set to 1 min by default. The documentation states that "Specifies how long an HTTP Request should be left unanswered before being evicted from the cache and being responded to with a Service Unavailable status code". If I increase this value I can see that I dont see any other duplicate values. So now I know why this is happening, but not sure as to why this is acting like this. Why am I getting another entry in the forward queue.

Answer 1

You are missing the HandleHttpResponse processor. Once you read the request from the HandleHttpRequest processor you must send a response back to the client (In this case postman). If postman doesn't get any response within 60 seconds it will retry due to the timeout.

See this flow built by me.

HandleHttpResponse processor's success queue will have the same flow file. Sometimes you might need to do a certain operation before you send the response to the client. In that case do not directly connect the HandleHttpResponse Processor directly to HandleHttpRequest processor. Once you have completed the operation, make sure you send the response to the client using HandleHttpResponse processor before the timeout.