How can I deconstruct a Spray API HTTPResponse?

Question

I'm using Spray API (spray-client) to hit an internal Solr URL, I want to be able to parse the response into a Scala case class.

If I just expect and HTTPResponse, I'm getting a value back, but when I try to marshal it into my case class, it fails (I can't produce a message other than null(), because I'm using matching and obviously not getting the right test case.)

I think some of my problem is that it's returning the data in the form of text/plain instead of application/json. When I expect HttpResponse instead of my case class,

val f: Future[HttpResponse] =
    (IO(Http) ? Get("http://1.2.3.4:8983/solr/collection1/select?q=*%3A*&wt=json")).mapTo[HttpResponse]

I get:

HttpResponse(200 OK,HttpEntity(text/plain; charset=UTF-8,
{
  "responseHeader":{"status":0,"QTime":65,"params":{"q":"*:*","wt":"json"}},
  "response":{"numFound":147437873,"start":0,"maxScore":1.0,"docs":
    [
      {"guid":"TLQ0jVlMYCXQrYkBIZHNXfMmifw+3","alias":["greg"],"_version_":1440942010264453120},
      {"guid":"TQsDY1ZG7q+Ne5e6F7qAUhFyomSH9","_version_":1440942010296958976},
      {"guid":"TzWB5grOBAJJZcAQDo2k9xBUVGPFr","alias":["spark"],"_version_":1440942010298007552},
      {"guid":"T0judCG4UI9RYqDDQVcn+gyZEU7Bb","alias":["zombie"],...),List(Connection: close, Content-Type: text/plain; charset=UTF-8),HTTP/1.1)

But when I change that to expect my case class, I can't match. So, how can I marshal the data it returns into a Scala case class? Here's what I have tried:

case class SolrParams(q: String, wt: String)
case class SolrResponseHeader(status: String, qtime: String, params: SolrParams)
case class SolrDoc(guid: String, alias: List[String], version: String)
case class SolrResponse(numFound: Long, start: Long, maxScore: String, docs: List[SolrDoc])

case class SolrApResult(responseHeader: SolrResponseHeader, response: SolrResponse)

object SolrJsonProtocol extends DefaultJsonProtocol {
  implicit val paramsFormat = jsonFormat2(SolrParams)
  implicit val responseHeaderFormat = jsonFormat2(SolrResponseHeader)
  implicit val docFormat = jsonFormat3(SolrDoc)
  implicit val responseFormat = jsonFormat4(SolrResponse)
  implicit def solrApiResultFormat = jsonFormat2(SolrApiFullResult)
}

...

val f: Future[SolrApiResult] =
    (IO(Http) ? Get("http://1.2.3.4:8983/solr/collection1/select?q=*%3A*&wt=json")).mapTo[SolrApiResult]

Which gives me no match in an f onComplete ... structure. Could the issue be that my case classes aren't matching what's being returned, and if so, what suggestions do you have to troubleshoot it better?

I've been all over the docs and they're either incomplete or a bit dated, plus I'm new at this game so that's not helping either.

Answer 1

The main issue I see with your code is that you are expecting mapTo to automatically unmarshall the http response body into your case class structure. The mapTo method belongs to the Future class and has no idea about json unmarshalling. You use mapTo within an Akka request (which ? is doing under the hood) to basically "cast" a Future[Any] (which is the default response type from ?) into the type that will actually be returned. This has nothing to do with json unmarshalling. I believe in order to achieve what you want, you should create a pipeline for spray that will both send the request and unmarshall the response. The documentation showing examples of that are here. I think for your example, it would look something like this (code may not be 100% right, just trying to show general flow):

val pipeline = sendReceive ~> unmarshall[SolrApiResult]
val response:Future[SolrApiResult] = pipeline(Get("http://1.2.3.4:8983/solr/collection1/select?q=*%3A*&wt=json"))