Ruby/Rack/Sinatra encoding of a MessagePack POST

I am trying to build an API with Sinatra and MessagePack but I am about to shoot myself :)

So I am using curl to do a POST:

curl -X POST -H "Content-Type: application/x-msgpack" --data-binary '\x82\xA4uuid\xBD8asd76a-a8s7a6d87-asd76as8d76\xABcampaign_id\xA12' http://localhost:9393/

Now in Sinatra I just do:

MessagePack.unpack request.body.read

And this leads to the following error:

MessagePack::MalformedFormatError Exception: extra bytes follow after a deserialized object

Because request.body.read is returning:

"\\x82\\xA4uuid\\xBD8asd76a-a8s7a6d87-asd76as8d76\\xABcampaign_id\\xA12"

instead of
"\x82\xA4uuid\xBD8asd76a-a8s7a6d87-asd76as8d76\xABcampaign_id\xA12"

I tried everything I could think of like force_encoding(Encode::BINARY) and other stupid things. And I am not sure who is causing the problem ruby, rack or sinatra?

Solution

The problem is what you expect curl --data-binary to do from the command line. It does not process '\x82' into a byte value using Ruby-like syntax. It posts the characters as-is (which looks like "\\x82" if you inspect the string in Ruby).

Use the @filename syntax for curl instead, and save a data file that you have generated using MessagePack instead (make sure to set the mode to 'wb' in Ruby):

curl -X POST -H "Content-Type: application/x-msgpack" --data-binary @test.dat http://localhost:9393/

Ruby to create test file:

msg = MessagePack.pack( 
  "uuid" => "8asd76a-a8s7a6d87-asd76as8d76", 
  "campaign_id" => "2"
) 
File.open( 'test.dat', 'wb' ) do |file|
  file.write( msg )
end