Spring ControllerAdvice does not return response body?

I have the following ControllerAdvice, that handles JsonParseException (I use Spring and Jackson)

@ControllerAdvice
public class ControllerExceptionHandler  extends ResponseEntityExceptionHandler {

  @ExceptionHandler(JsonParseException.class)
  public ResponseEntity<Object> handleInvalidJson(JsonParseException ex, WebRequest request){
    Map<String,Object> body = new LinkedHashMap<>();
    body.put("timestamp", LocalDateTime.now());
    body.put("message","Invalid Json");

    return new ResponseEntity(body, HttpStatus.BAD_REQUEST);
 }
}

Fore some reason, It doesn't work when I send a bad json request to the server, only returns 400. When I change the HttpStatus, it still returns 400 so it seems like the advice doesn't really run.

ResponseEntityExceptionHandler already implements a lot of different ExceptionHandlers. HttpMessageNotReadableException is one of them:

else if (ex instanceof HttpMessageNotReadableException) {
            HttpStatus status = HttpStatus.BAD_REQUEST;
            return handleHttpMessageNotReadable((HttpMessageNotReadableException) ex, headers, status, request);
        }

Simply remove the inheritance:

@ControllerAdvice
public class TestExceptionHandler {

    @ExceptionHandler(JsonParseException.class)
    public ResponseEntity<Map<String,Object>> handleInvalidJson(JsonParseException ex, WebRequest request){
        Map<String,Object> body = new LinkedHashMap<>();
        body.put("timestamp", LocalDateTime.now());
        body.put("message","Invalid Json");

        return new ResponseEntity<>(body, HttpStatus.I_AM_A_TEAPOT);
    }
}