Why shouldn't custom methods use the URL for transferring data?

TL,DR; When implementing custom methods, "the HTTP configuration [...] must use the body:* clause and all remaining request message fields shall map to the HTTP request body.". Why?

I have a problem with Google's API Design Guide which I'm attempting to follow with gRPC with Cloud Endpoints.

The HttpRule is used to transcode HTTP/JSON to gRPC. The HttpRule reference states:

Note that when using * in the body mapping, it is not possible to have HTTP parameters, as all fields not bound by the path end in the body.

[...] The common usage of * is in custom methods which don't use the URL at all for transferring data.

...an opinion also repeated in Google's Custom Methods documentation and reinforced with Google's API Linter,

When using a named representation in the body mapping there is a well defined space left to add metadata in the form of querystring parameters; E.g. for pagination, links, deprecation warnings, error messages).

service Messaging {
  rpc UpdateMessage(UpdateMessageRequest) returns (Message) {
    option (google.api.http) = {
      put: "/v1/messages/{message_id}"

      // A named reference makes it possible to use querystring params
      // and the HTTP body.
      body: "data"
    };
  }
}
message UpdateMessageRequest {
  message Data {
    string foo = 1;
    string bar = 2;
    string baz = 3;
  }

  // mapped to the URL as querystring params
  bool format = 1;
  string revision = 2;

  // mapped to the body
  Data data = 3;
}

This allows for an HTTP PUT request to /v1/messages/123456?format=true&revision=2 with a body

foo="I am foo"
bar="I am bar"
baz="I am baz"

Since the mapping binds body to the type UpdateMessageRequest.Data, the remaining fields end up in the querystring. This is the approach used in standard methods, but not with custom) methods.

Custom methods must map body to *. The same API with a custom method would be

service Messaging {
  rpc UpdateMessage(UpdateMessageRequest) returns (Message) {
    option (google.api.http) = {
      put: "/v1/messages/{message_id}"

      // Every field not bound by the path template should be
      // mapped to the request body.
      body: "*"
    };
  }
}
message UpdateMessageRequest {
  message Data {
    string foo = 1;
    string bar = 2;
    string baz = 3;
  }

  // mapped to the body
  bool format = 1;
  string revision = 2;
  Data data = 3;
}

If the same metadata is used across both standard and custom) methods, it must be added either as querystring params, or placed in the body.

For example, an Angular app would use HttpParams

// standard method
const params = new HttpParams().append('format', true).append('revision', 2);
const request = {
  foo: "I am foo",
  bar: "I am bar",
  baz: "I am baz",
}
this.http.post<Document>(url, request, {params});

However, a custom method requires the client to place everything in the body:

// custom method
const request = {
  format: true,
  revision: 2,
  data: {
    foo: "I am foo",
    bar: "I am bar",
    baz: "I am baz",
  },
}
this.http.post<Document>(url, request);

Question: What is the reason for this?

Great question.

For reference, I wrote the AIP on this topic as well as the lint rule, and am also the current maintainer of the design guide that you referenced.

First off, I will mention that our latest guidance (linked above) specifically says should rather than must for this. In other words, it is the right thing to do the majority of the time, but there may be exceptions. Nothing in the gRPC transcoding implementation stops you from using a different body -- we tell you to use * for custom methods, but we do not place any technical barriers against doing something else.

I can think of a couple of good "exception cases" where a body other than * might make sense. The first would be a custom method that is modeled on one of the standard methods, but should be custom for some reason. The second would be if a custom method accepted a full resource, and wanted to set the body to that resource. This would make that method consistent with Create and Update, which obviously has value to the API's users.

If you have a case where you have a clear argument for using something else as the body (particularly if that something is the resource itself), by all means, use a different body and tell the linter to be quiet. We wrote "should" for a reason.

You also asked: Why do we have that recommendation in the first place?

There are a few reasons. The biggest one is that the exceptional cases described above are rare. I do hundreds of API reviews internally, and I can not actually think of one off the top of my head (which is not to say they do not exist). The majority of the time, the best thing for users is for the request message to mirror the HTTP payload.

Another reason is a key limitation: assigning a particular field to be the body limits what you can add outside of that field, since query strings are limited in what they can represent in both type (just primitives) and quantity (URI length constraints). Because altering the body later constitutes a breaking change, this ties your hands somewhat. That might be fine for your use case, obviously, but it is important to note.

Anyway, I hope that helps -- oh, and thanks for using my stuff. :-)