Pulling files from MultipartFormData in memory in Play2 / Scala

Question

I'm currently using the following with Play2/Scala using the FileUploader Javascript utility to upload a file to my server:

def fileUploader = Action(parse.multipartFormData) { request =>
  request.body.file("qqfile").map { picture =>
    import java.io.File
    val filename = picture.filename 
    val contentType = picture.contentType
    picture.ref.moveTo(new File("/tmp",filename))
    Ok(Json.toJson(Map( "success" -> "true" )))
  }.getOrElse {
    Ok(Json.toJson(Map( "error" -> "error occured")))
  }
}

I'm only dealing with small files (<10MB) and I want to use casbah to write those files directly into a Mongo Document or GridFS using the Mongo drivers. I realize I could just read the saved file from disk, but is there a way to handle this all from memory without buffering the file on disk first?

The play documentation here recommends writing a custom BodyParser (http://www.playframework.com/documentation/2.1.0/ScalaFileUpload) but there doesn't seem to be any documentation on how to go about writing one. It wasn't clear how the API/implementation worked from the Scaladocs. I tried looking for the MultiPartFormData source code to see how it worked, but I can't seem to find it in their Git repo:

https://github.com/playframework/Play20/tree/master/framework/src/play/src/main/scala/play/api/mvc

I've searched quite a bit, but can't seem to find a good example.

Answer 1

Following on from Matt's answer (hi, Matt!), I wound up needing to tweak that very slightly for Play 2.8 (I suspect the API evolved a little further):

  def byteStringFilePartHandler: FilePartHandler[ByteString] = {
    case FileInfo(partName, filename, contentType, dispositionType) =>
      Accumulator(Sink.fold[ByteString, ByteString](ByteString()) { (accumulator, data) =>
        accumulator ++ data
      }.mapMaterializedValue(fbs => fbs.map(bs => {
        FilePart(partName, filename, contentType, bs)
      })))
  }

  def multipartFormDataAsBytes: BodyParser[MultipartFormData[ByteString]] =
    controllerComponents.parsers.multipartFormData(byteStringFilePartHandler)

Since my use case is uploading text files, I fetch that out of the resulting request with:

    val body: String = request.body.files.head.ref.utf8String

(A less quick-and-dirty bit of code would use headOption there, just to be safe.)

Answer 2

The Play API has changed a decent amount since this was posted. I had a similar use case where I didn't want a temp file and translated the above into the following, which seems to work with Play 2.6 in case anyone needs this:

def byteStringFilePartHandler: FilePartHandler[ByteString] = {
    case FileInfo(partName, filename, contentType) =>
      Accumulator(Sink.fold[ByteString, ByteString](ByteString()) { (accumulator, data) =>
        accumulator ++ data
      }.mapMaterializedValue(fbs => fbs.map(bs => {
        FilePart(partName, filename, contentType, bs)
      })))
}

def multipartFormDataAsBytes: BodyParser[MultipartFormData[ByteString]] =
  playBodyParsers.multipartFormData(byteStringFilePartHandler)

Using it in a controller make sure you inject PlayBodyParsers and provide an ExecutionContext, imports etc below:

import akka.stream.scaladsl.Sink
import akka.util.ByteString
import javax.inject._
import play.api.libs.streams.Accumulator
import play.api.mvc.MultipartFormData.FilePart
import play.api.mvc._
import play.core.parsers.Multipart.{FileInfo, FilePartHandler}
import scala.concurrent.ExecutionContext


@Singleton
class HomeController @Inject()(cc: ControllerComponents, playBodyParsers: PlayBodyParsers)
                              (implicit ec: ExecutionContext) extends AbstractController(cc) {

  def index = Action(multipartFormDataAsBytes) { request =>
    request.body.file("image").foreach((image) => {
      val arr = image.ref.toByteBuffer.array()
      println(arr)
    })
    Ok("got bytes!")
  }
}

Answer 3

Untested The Multipart object of the BodyParsers does a lot of work for us. The first thing we need to do write a handler for the FilePart. I assume here that you want the file parts an Array[Byte].

def handleFilePartAsByteArray: PartHandler[FilePart[Array[Byte]]] =
  handleFilePart {
    case FileInfo(partName, filename, contentType) =>
      // simply write the data to the a ByteArrayOutputStream
      Iteratee.fold[Array[Byte], ByteArrayOutputStream](
        new ByteArrayOutputStream()) { (os, data) =>
          os.write(data)
          os
        }.mapDone { os =>
          os.close()
          os.toByteArray
        }
  }

The next step is to define your body parser:

def multipartFormDataAsBytes:BodyParser[MultipartFormData[Array[Byte]]] = 
  multipartFormData(handleFilePartAsByteArray)

Then, in order to use it, specify it at you Action:

def fileUploader = Action(multipartFormDataAsBytes) { request =>
  request.body.files foreach {
    case FilePart(key, filename, contentType, bytes) => // do something
  }
  Ok("done")
}

Some types and methods in the above pieces of code are a bit hard to find. Here is a complete list of imports in case you need it:

import play.api.mvc.BodyParsers.parse.Multipart.PartHandler
import play.api.mvc.BodyParsers.parse.Multipart.handleFilePart
import play.api.mvc.BodyParsers.parse.Multipart.FileInfo
import play.api.mvc.BodyParsers.parse.multipartFormData
import play.api.mvc.MultipartFormData.FilePart
import play.api.libs.iteratee.Iteratee
import java.io.ByteArrayOutputStream
import play.api.mvc.BodyParser
import play.api.mvc.MultipartFormData