load-testinggatling

How to write custom feeder for rest json template


I wrote a simple "tutorial" application that offers a REST interface called "world endpoint" which you can use to add inhabitants etc. A sample json request to do so might look like this:

{
   "name": "Sergio Gonzales",
   "age": "34",
   "languages": [
      { "id": "119" },
      { "id": "14" }
   ],
   "homeland": { "id": "121" },
   "hometown": { "id": "155" }
}

I'd like to add a gatling load test to test the "creation" of new inhabitants. To get the data, I have three different sources:

  1. first_names.csv
  2. last_names.csv
  3. country_capital_language.csv

First two will be used to create random names (its of course no highly sophisticated approach to create meaningful data). Last one will be used to choose ids for homeland, hometown and the native language. Besides I'll choose additional 0 - 3 languages randomly.

I'd assume that I have to write an own feeder for that but regretfully the documentation for custom feeder seems to be gone since release 2.x.x.

What would be a good approach to write the feeder? My first thought was to load the csv data directly like this:

Source.fromInputStream(getClass.getResourceAsStream("/names/first_names.csv")).getLines.toSet

Not sure if using csv("first_names.csv") would be a better approach so?

Besides I also do not know, how I can replace the "languages" section in the json by dynamically created data? Is it possible, to pass the list of language ids and it gets automatically translated into valid json array?

UDPATE

Here is my first working version. It has flaws but it basically does what I want. If anyone has recommendations how to enhance, please do not hesitate (I am pretty new to scala).

package com.u6f6o.apps.hazelnate.load.scenario

import io.gatling.core.Predef._
import io.gatling.core.feeder.Record
import io.gatling.http.Predef._

import scala.concurrent.forkjoin.ThreadLocalRandom

class Insert100kInhabitants extends Simulation {
  val random = ThreadLocalRandom.current
  val footprints = csv("data/footprints.csv").records
  val forenames = csv("data/forenames.csv").records
  val surnames = csv("data/surnames.csv").records

  val httpConf = http
    .baseURL("http://localhost:4567")
    .acceptHeader("application/json")
    .doNotTrackHeader("1")

  val scn = scenario("Insert100kInhabitants").repeat(10000){
    exec{ session =>
      val footprintRecords = chooseRandomly(footprints, 5)
      session.setAll(
        "forename" -> chooseRandomly(forenames).getOrElse("forename", ""),
        "surname" -> chooseRandomly(surnames).getOrElse("surname", ""),
        "age" -> random.nextInt(1, 110),
        "hometown" -> footprintRecords.head.getOrElse("city", ""),
        "homeland" -> footprintRecords.head.getOrElse("country", ""),
        "languages" -> footprintRecords.map{ x => x.getOrElse("language", "")}
      )
    }
    .exec(http("insert100kInhabitants")
      .post("/world/inhabitants")
      .body(StringBody( session => generateJson(session))).asJSON
    )
  }

  setUp(
    scn.inject(atOnceUsers(10))
  ).protocols(httpConf)

  def generateJson(session:Session) : String = {
    s"""{
      |   "name": "${session("forename").as[String]} ${session("surname").as[String]}",
      |   "age": "${session("age").as[String]}",
      |   "hometown": "${session("hometown").as[String]}",
      |   "homeland": "${session("homeland").as[String]}",
      |   "languages": [
      |     ${session("languages").as[Seq[String]].map{ x => s"""{ "id": "${x}" }"""}.mkString(", ")}
      |   ]
      |}""".stripMargin
  }

  def chooseRandomly(pool:IndexedSeq[Record[String]]) : Record[String] = {
    pool(random.nextInt(pool.length))
  }

  def chooseRandomly(pool:IndexedSeq[Record[String]], maxLength:Int) : IndexedSeq[Record[String]] = {
    for (i <- 1 to random.nextInt(1, maxLength)) yield pool(random.nextInt(pool.length))
  }
}

Solution

  • For first name and last name, use simple Feeders.

    For more complex data injection logic, don't use a Feeder. Write a custom exec(function) where you manually pick records and set them into the Session. You can still use Gatling's csv parser in order to load the data:

    val records: Seq[Map[String, Any]] = csv("country_capital_language.csv").records
    

    As you want a dynamic number of languages, you won't be able to use a Gatling EL template. You'll have to manually craft your request bodies, see documentation.