Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get stream from file upload in openapi

I have a backend application (Java 8) and an Angular frontend. Now, I added a file upload using multipart/formdata. We use openapi to generate the api code. Here's the part of the code used to generate it:

"post": {
   ...
   "consumes" : [
      "multipart/form-data"
   ],
   "produces": [
      "application/json"
   ],
   "parameters": [
      {
         "name": "updateFile",
         "in": "formData",
         "type": "file",
         "required": true,
      }
   ],
   ...
}

And the frontend prepares the file as such:

  const formData = new FormData();
  formData.append('updateFile', updateFile);

This actually works quite well and provides us a Part-Object. Unfortunately the machine this code will run on in production is quite weak and the uploaded file will be too large to be handled in the memory of it (~130mb). So, I need a workaround for this. I think accessing the incoming stream would be an elegant way to write the file to some temp directory "on-the-fly" and I've come across code samples doing exactly that. I can't figure out how to configure my endpoint to provide a stream instead of the "finished" Part-Object.

Unfortunately this project uses a quite old openapi version 2.

like image 607
fhueser Avatar asked Apr 19 '21 09:04

fhueser


People also ask

How do you send a multipart file in request body in swagger?

You start with the requestBody/content keyword. Then, you specify the media type of request data. File uploads typically use the multipart/form-data media type, and mixed-data requests usually use multipart/mixed . Below the media type, put the schema keyword to indicate that you start describing the request payload.

What is multipart file upload?

Multipart upload allows you to upload a single object as a set of parts. Each part is a contiguous portion of the object's data. You can upload these object parts independently and in any order. If transmission of any part fails, you can retransmit that part without affecting other parts.

How do I upload a file to OpenAPI?

File Upload. In OpenAPI 3.0, you can describe files uploaded directly with the request content and files uploaded with multipart requests. Use the requestBody keyword to describe the request payload containing a file. Under content, specify the request media type (such as image/png or application/octet-stream ).

Which version of OpenAPI do you use for OAS?

OAS 3 This guide is for OpenAPI 3.0. If you use OpenAPI 2.0, see our OpenAPI 2.0 guide. In OpenAPI 3.0, you can describe files uploaded directly with the request content and files uploaded with multipart requests.

Is there an API for uploading and downloading files using loopback?

It’s a common requirement for API applications to support file upload and download. This page describes how to expose REST APIs for uploading and downloading files using LoopBack 4. It also illustrates how to build a simple Web UI to interact with such APIs.

What is the processing part of streams?

The processing part makes streams particularly charming as it makes dealing with bigger files more efficient and lives the spirit of node's event loop unblocking i/o magic. To visualize streams, consider the following example. You have a single file with a size of 4 gb. When processing this file, it is loaded into your computers memory.


Video Answer


1 Answers

The workaround I've found now is taking the ServletInpuStream directly from the HttpServletRequest and leaving the body in the API description empty to prevent the stream from being consumed before I actively do so. This of course forces me to parse the content myself. I found MultipartStream of apache.commons to be very helpful for this.

This solution feels a bit hacky but is at least viable until we updated to a more recent version that directly supports something like that.

like image 65
fhueser Avatar answered Oct 23 '22 00:10

fhueser