Use IdentityServer3.AccessTokenValidation in ASP.Net web api to talk to IDS4

We have setup Identity server 4 with .Net Core 2.1. We have ASP.Net Web API on .Net 4.6. This cannot be migrated to Core. I have installed IdentityServer3.AccessTokenValidation in this API project together with OWN and other neeeded packages. I can use a client and get the access token from the IdentityServer. When the client uses that token with the API, I get 401. I have searched the web for solutions but cannot find anything that I can use to make this work. I tried this - https://github.com/IdentityServer/IdentityServer3.AccessTokenValidation/issues/164 but did not work. I cannot find any samples. So my question is is it not possible to use IdentityServer3.AccessTokenValidation to talk to IDS4?

Do I need to setup IDS3 instead?

UPDATE

I turned on verbose logging and got the following

Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request starting HTTP/1.1 POST http://localhost:44333/connect/introspect application/x-www-form-urlencoded 70
IdentityServer4.Hosting.IdentityServerMiddleware:Information: Invoking IdentityServer endpoint: IdentityServer4.Endpoints.IntrospectionEndpoint for /connect/introspect
IdentityServer4.Validation.ApiSecretValidator:Error: API validation failed.
IdentityServer4.Endpoints.IntrospectionEndpoint:Error: API unauthorized to call introspection endpoint. aborting.
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request finished in 26.5186ms 401 
Microsoft.AspNetCore.Server.Kestrel:Information: Connection id "0HLIJ2C0UKBLN", Request id "0HLIJ2C0UKBLN:00000008": the application completed without reading the entire request body.

Below is my API code in Configuration method

        var config = new HttpConfiguration();
        config.MapHttpAttributeRoutes();

        app.UseIdentityServerBearerTokenAuthentication(new IdentityServerBearerTokenAuthenticationOptions
        {
            Authority = "https://localhost:44333",
            RequiredScopes = new[] { "api" },

            ClientId = "api",
            ClientSecret = "secret",

            //DelayLoadMetadata = true
        });

        app.UseWebApi(config);

Code on IDS4 side for the ApiResource.

    public static IEnumerable<ApiResource> GetApiResources()
    {
        return new List<ApiResource>
        {
            new ApiResource("api", "MyApi") { ApiSecrets = { new Secret("secret".Sha256()) } }
        };
    }

UPDATE 2

Yes I have enabled Trace. After bit more of playing around with the code and reinstalling the packages, I now get a different error.Request is made to /connect/accesstokenvalidation. This endpoint is not available in IDS4

Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request starting HTTP/1.1 POST http://localhost:44333/connect/accesstokenvalidation application/x-www-form-urlencoded 70 Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request finished in 3.405ms 404 Microsoft.AspNetCore.Server.Kestrel:Information: Connection id "0HLIJ9RBG591K", Request id "0HLIJ9RBG591K:0000000A": the application completed without reading the entire request body.

OK Finally I managed to solve it after 3 days of headache.

connect/accesstokenvalidation was because of old version of IdentityServer3.AccessTokenValidation. I updated this to version 2.15.1 and also downgarded System.IdentityModel.Tokens.Jwt to 4.0.2.206221351

I also added scopes to ApiResource as below. However this might not be required. I have not tested without it yet.

    public static IEnumerable<ApiResource> GetApiResources()
    {
        return new List<ApiResource>
        {
            new ApiResource("my_api", "SmartBiz Api")
            {
                ApiSecrets = { new Secret("secret".Sha256()) },
                Scopes =
                {
                    new Scope("the_api")
                }
            }
        };
    }

Thanks for your help Rosdi Kasim