How to cancel previous request in FastAPI

Question

I have a FastAPI app with an end point that may have to do a long computation. As a replacement of the actual code let me use this mock example, in which I am trying to keep the features that I think are relevant.

from fastapi import FastAPI
from fastapi.responses import JSONResponse
import subprocess

app = FastAPI()

@app.put("/expensive")
def expensive():
  command = "for x in $(seq 1 10000); do echo ${x}; done"
  subprocess.run(command, shell=True, check=True)
  return JSONResponse(content=True)


if __name__ == "__main__":
  import uvicorn
  uvicorn.run(
    "deleteme:app",
    host='0.0.0.0',
    port=12345,
    reload=True
  )

As in the mock example, the actual code run a third party software that does a computation that could, in some cases, take a long time.

When I send, with Postman, a PUT request, the app will start printing the natural numbers from 1 to 10000. I can cancel the request in Postman. However, the computation in the app will continue.

Question: I would like the user to be able to have a mechanism to cancel a specific request made to http://0.0.0.0:12345/expensive. What are ways to do this?

My lack of experience with Web APIs also causes that I am not finding the right key words to search about the subject. For example, combinations of fastapi request abort, or web api request cancel, don't seem to lead to posts discussing this topic.

A mechanism that I can think about could be:

1. Send in the initial request an ID request_id.
2. When the third party software is run, get its PID and store the key:value pair request_id:PID.
3. Have a second end point to receive cancel requests. The user would submit the request_id of the request that they would like to cancel.
4. Use the request_id to then kill the subprocess with the corresponding PID.

This seem convoluted.

Answer 1

There's a way to "cancel"(or kill) the request(or subprocess). But it isn't easy.

First, Canceling the actual request is impossible because it is a FastAPI process. I don't know how exactly FastAPI runs internally, but it's a low-level story.

Second, killing the subprocess is possible. You can find the subprocess's PID and the only thing left to do is kill the process. It's not fancy, though.

There are lots of alternative solutions.

1. Set timeout: subprocess.run() has timeout parameter. It will raise subprocess.TimeoutExpired

2. Use BackgroundTasks: FastAPI(Starlette, actually) has nice feature, Background Tasks

3. Separate your service with a 3rd party: It's recommended if your process(or another app) takes a very long time. Celery is for doing this. Many best practices are there, and I think it is a proper way what you think.

Answer 2

In my experience, long running requests have always resulted in one problem or another. A few 'heavy' requests should not be allowed to prevent the workers from serving numerous small requests. I would suggest a polling based mechanism with an ETA feature.

1. API to initiate jobs. Returns a JOB-ID (a random string that is mapped to PID and other resources internally) and an expected time taken for processing.
2. API to get the job results. Returns either results or error with details on how long more to wait or if the job failed.
3. API to kill a job and free resources.
4. A cleanup daemon process. Since we do not want to keep unfetched job results for ever.

Another option is to switch to websockets. With a pingpong we can detect the connection failure and stop the process when needed.