How to write custom feeder for rest json template

Question

I wrote a simple "tutorial" application that offers a REST interface called "world endpoint" which you can use to add inhabitants etc. A sample json request to do so might look like this:

{
   "name": "Sergio Gonzales",
   "age": "34",
   "languages": [
      { "id": "119" },
      { "id": "14" }
   ],
   "homeland": { "id": "121" },
   "hometown": { "id": "155" }
}

I'd like to add a gatling load test to test the "creation" of new inhabitants. To get the data, I have three different sources:

1. first_names.csv
2. last_names.csv
3. country_capital_language.csv

First two will be used to create random names (its of course no highly sophisticated approach to create meaningful data). Last one will be used to choose ids for homeland, hometown and the native language. Besides I'll choose additional 0 - 3 languages randomly.

I'd assume that I have to write an own feeder for that but regretfully the documentation for custom feeder seems to be gone since release 2.x.x.

What would be a good approach to write the feeder? My first thought was to load the csv data directly like this:

Source.fromInputStream(getClass.getResourceAsStream("/names/first_names.csv")).getLines.toSet

Not sure if using csv("first_names.csv") would be a better approach so?

Besides I also do not know, how I can replace the "languages" section in the json by dynamically created data? Is it possible, to pass the list of language ids and it gets automatically translated into valid json array?

UDPATE

Here is my first working version. It has flaws but it basically does what I want. If anyone has recommendations how to enhance, please do not hesitate (I am pretty new to scala).

package com.u6f6o.apps.hazelnate.load.scenario

import io.gatling.core.Predef._
import io.gatling.core.feeder.Record
import io.gatling.http.Predef._

import scala.concurrent.forkjoin.ThreadLocalRandom

class Insert100kInhabitants extends Simulation {
  val random = ThreadLocalRandom.current
  val footprints = csv("data/footprints.csv").records
  val forenames = csv("data/forenames.csv").records
  val surnames = csv("data/surnames.csv").records

  val httpConf = http
    .baseURL("http://localhost:4567")
    .acceptHeader("application/json")
    .doNotTrackHeader("1")

  val scn = scenario("Insert100kInhabitants").repeat(10000){
    exec{ session =>
      val footprintRecords = chooseRandomly(footprints, 5)
      session.setAll(
        "forename" -> chooseRandomly(forenames).getOrElse("forename", ""),
        "surname" -> chooseRandomly(surnames).getOrElse("surname", ""),
        "age" -> random.nextInt(1, 110),
        "hometown" -> footprintRecords.head.getOrElse("city", ""),
        "homeland" -> footprintRecords.head.getOrElse("country", ""),
        "languages" -> footprintRecords.map{ x => x.getOrElse("language", "")}
      )
    }
    .exec(http("insert100kInhabitants")
      .post("/world/inhabitants")
      .body(StringBody( session => generateJson(session))).asJSON
    )
  }

  setUp(
    scn.inject(atOnceUsers(10))
  ).protocols(httpConf)

  def generateJson(session:Session) : String = {
    s"""{
      |   "name": "${session("forename").as[String]} ${session("surname").as[String]}",
      |   "age": "${session("age").as[String]}",
      |   "hometown": "${session("hometown").as[String]}",
      |   "homeland": "${session("homeland").as[String]}",
      |   "languages": [
      |     ${session("languages").as[Seq[String]].map{ x => s"""{ "id": "${x}" }"""}.mkString(", ")}
      |   ]
      |}""".stripMargin
  }

  def chooseRandomly(pool:IndexedSeq[Record[String]]) : Record[String] = {
    pool(random.nextInt(pool.length))
  }

  def chooseRandomly(pool:IndexedSeq[Record[String]], maxLength:Int) : IndexedSeq[Record[String]] = {
    for (i <- 1 to random.nextInt(1, maxLength)) yield pool(random.nextInt(pool.length))
  }
}

Answer 1

For first name and last name, use simple Feeders.

For more complex data injection logic, don't use a Feeder. Write a custom exec(function) where you manually pick records and set them into the Session. You can still use Gatling's csv parser in order to load the data:

val records: Seq[Map[String, Any]] = csv("country_capital_language.csv").records

As you want a dynamic number of languages, you won't be able to use a Gatling EL template. You'll have to manually craft your request bodies, see documentation.

Answer 2

By now Gatling can handle all that stuff on its own.

Note: I haven't tested the simplified code examples here. So forgive me for any typos.

1. Templates in JSON: https://groups.google.com/forum/#!topic/gatling/ggR6L4ukmqU. Be aware that the column names in the CSV files must match the names in the template and you need to use ElFileBody (http://gatling.io/docs/2.2.1/http/http_request.html).
   
   myTemplate.json

   {
    "name": "${firstName} ${lastName}",
    "age": "34",
    "languages": [
       { "id": "119" },
       { "id": "14" }
    ],
    "homeland": { "id": "121" },
    "hometown": { "id": "155" }
   }
   

   Expected content of e.g. names.csv: http://gatling.io/docs/2.2.1/session/feeder.html

   firstName, lastName
   Miguel, Prado
   Dexter, Morgan
   Deborah, Mrogan
   
   

2. Random CSV Feeder

   val scn = scenario("Insert100kInhabitants").repeat(10000){
   feed(csv("names.csv").random)
     .exec(http("insert100kInhabitants")
       .post("/world/inhabitants")
       .body(ElFileBody("myTemplate.json").asJSON
     )
   }