unit-testingavrocascadingscalding

HowTo mock PackedAvroSource in scadling JobTest


I'm trying to mock PackedAvroSource, no luck. Here is an exception:

unable to resolve argument selector: [{1}:'bytes'], with incoming: [{1}:0]

it happens when I try to access 'byte field in map taken from avro. Map func whic converts bytes expects to get field named 'bytes

Here is a test code snippet

test("My test"){
    JobTest(classOf[PackedAvroSourceJob].getName)
      .arg("input", "input")
      .arg("output", "output")

      .source(PackedAvroSource[ByteBuffer]("input"), createInput)
      .sink(PackedAvroSource[ByteBuffer]("output")) { buffer: mutable.Buffer[(Byte)] =>
      println("hERE!!!")
    }
      .run
  }

What is the right way to pass schema 'bytes to that PackedAvroSource?


Solution

  • In your Job, when reading PackedAvroSource, use Fields.FIRST for the ByteBuffer field. This will work for both the job and the test.