[SOLVED] Restrict length of log output in Rails

Restrict length of log output in Rails

I have an API that accepts base64 encoded images. The requests come in looking like:

{
  image: "ijt41ig8ahbha8oij4j1tjletc"
}

The base64 encoded strings can be very large, sometimes 10k - 100k characters. When a request is sent with improperly formatted JSON, Active Dispatch correctly raises an error and logs the request:

ActionDispatch::ParamsParser::ParseError (765: unexpected token at '{
    "image": "iVlBWsBo4OQT1Mm7AZbIBV3h8mwtvLdakElJ6U1ksbQsl3URJ
    iVlBWsBo4OQT1Mm7AZbIBV3h8mwtvLdakElJ6U1ksbQsl3URJ
    iVlBWsBo4OQT1Mm7AZbIBV3h8mwtvLdakElJ6U1ksbQsl3URJ
    iVlBWsBo4OQT1Mm7AZbIBV3h8mwtvLdakElJ6U1ksbQsl3URJ"
}'):
actionpack (5.0.5) lib/action_dispatch/http/parameters.rb:88:in `rescue in parse_formatted_parameters'
actionpack (5.0.5) lib/action_dispatch/http/parameters.rb:82:in `parse_formatted_parameters'
actionpack (5.0.5) lib/action_dispatch/http/request.rb:354:in `block in POST'

etc

In this case the error is caused by new lines in the encoding.

My question is, what is a good way to restrict the logging to not display the sometimes 50k char long image string? This is filling up my log files and making looking through logs difficult. I do want the error logged, I just want to slice the string to maybe show the first 255 chars instead of the massive image.

I know that ActionDispatch has some nice ways of filtering attributes (for instance passwords), but that assumes that the parameters have been parsed into a hash, which in the case of a JSON error they have not.

Solution

I ended up creating a custom formatter and just manipulating the string directly

class CustomFormatter < Logger::Formatter

  def call(severity, time, program_name, message)
    if message =~ /"image": /
      message.gsub!(/(?<="image": )".+"/m) { |image| image[0..255]  + "_TRUNCATED_" }
    end

    super
  end

end

Then just use that formatter in your config file

  config.logger = Logger.new(STDOUT).tap do |logger|
    logger.formatter = CustomFormatter.new
  end