reCaptcha enterprise assessment returns Forbidden

Background - I'm implementing "reCAPTCHA Enterprise REST API, using API keys for authentication". I created a project and a reCaptcha test key (site key) at https://console.cloud.google.com/security/recaptcha using localhost as domain but domain verification turned off for now (6L***...***vw). I create an API key at https://console.cloud.google.com/apis/credentials.

My javascript has this for my script loader

scripts = ["https://www.google.com/recaptcha/enterprise.js?render=6L***...***vw"];
loadScripts(0);

and then when you sign in it triggers this

var recaptchResponse;
grecaptcha.enterprise.ready(async () => {
    recaptchResponse = await grecaptcha.enterprise.execute('6L***...***vw', { action: 'LOGIN' });
    authSrv2.login(vm.emailAddress, vm.password, recaptchResponse, 'LOGIN').then(function (loginResult) {
...

which triggers an API call to my backend which is Web API in C# and runs RunAssessment

public class reCaptcha {
    public async Task<bool> RunAssessment(string token, string action) {
        // score based key, recaptcha key, enterprise key. site key
        string SiteKey = "6L***...***vw";
        string APIKey = "AI***...***So";

        string project = ConfigurationManager.AppSettings.Get("reCaptchaProject");
        string url = "https://recaptchaenterprise.googleapis.com/v1/projects/" + project + "/assessments?key=" + APIKey;
        string myJson = "{\"event\": { \"token\": \"" + token + "\", \"siteKey\": \"" + SiteKey + "\", \"expectedAction\": \"" + action + "\" } }";
        HttpResponseMessage response;
        using (var client = new HttpClient()) {
            try {
                ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
                response = await client.PostAsync(url, new StringContent(myJson, Encoding.UTF8, "application/json")).ConfigureAwait(false);
                if (response.IsSuccessStatusCode) {
                    // TODO parse results
                    var dynamicObject = JsonConvert.DeserializeObject<dynamic>(response.Content.ToString());
                    return true;
                } else {
                    return false;
                }
            } catch (Exception ex) {
                var y = ex;
                return false;
            }
        }
    }
}

response always returns a 403 Forbidden Any ideas?

UPDATE From fiddler - request header

request json

Response json

Turns out I was able to make this work by manually adding the referer header