Unauthorized client error when using domain wide delegation with Google API

We are using Google API Ruby client with Domain Wide Delegation to update users' contacts, parse emails to pair them with our DB etc. Before April 1st everything was working just fine, however, since then we get unauthorized_client error response.

Signet::AuthorizationError (Authorization failed.  Server message:)
{
  "error": "unauthorized_client",
  "error_description": "Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested."
}

What have I already tried?

I have tried updating google-api-ruby-client and googleauth gems to latest version. Nothing changed.

I have also tried (twice) generating new service account, enabling it for DWD and adding the needed scope in G Suite admin console with service account's client ID. I waited for 24 hours after adding scopes in admin console and nothing changed.

Note that everything was working fine in the past with the same settings.

Minimal code for test

Here is the minimal code that throws the error.

require 'googleauth'
require 'google/apis/gmail_v1'

creds = ENV['GOOGLE_SERVICE_ACCOUNT'] # JSON downloaded from cloud console
                                      # is saved in this ENV variable
creds_json = JSON.parse(creds)
creds_json_io = StringIO.new(creds_json.to_json)
auth = Google::Auth::ServiceAccountCredentials.make_creds(
  json_key_io: creds_json_io,
  scope: [Google::Apis::GmailV1::AUTH_GMAIL_MODIFY]
)
auth.sub = 'admin@slideslive.com' # without this line everything works fine
                                  # and I get the access token
auth.fetch_access_token

As far as I can tell we are doing everything according to available documentation. Does anyone have any suggestions, please?

Update

I have also tried creating a new Google Cloud Platform project under the same organization with only Service Account added but everything stays the same.

So I have finally figured out where the problem was. Among other scopes I also enabled these:

http://www.google.com/m8/feeds/contacts/ 
http://www.google.com/m8/feeds/groups/ 

Turns out Google probably invalidated them and they now require https instead of http.

It would be nice if they provided better error message, something like that entered scopes are invalid when authorizing them in Google Admin Console.

I found the mistake when trying to authorize normal OAuth2 Client ID for the same scopes as I used for the service account. The OAuth2 consent screen shows proper error message.

Update

Here is the checklist I created for troubleshooting service accounts errors:

1. Check that Domain Wide Delegation is enabled for service account in Google Cloud Console.
2. Check that you entered the proper scopes in Google Admin Console with service account's Client ID (not its email!). Make sure all the scopes are valid! They might become invalid during their lifetime. When you enter invalid scope you get unauthorized_client error when fetching access token even for valid scope.