[SOLVED] Why is Flurl response empty when fetching lyricsify.com?

Why is Flurl response empty when fetching lyricsify.com?

I want to use Flurl to query lyricsify.com but I always get an empty string as result.

With Insomnia, Powershell and other tools, I get the HTML response as expected.

[Fact]
public async void FlurlTest()
{
    string googleResponse = await "https://www.google.com".GetStringAsync();
    Debug.WriteLine($"googleResponse: {googleResponse}");
    // works as expected
    Assert.NotEmpty(googleResponse);
    
    string lyricsifyResponse = await "https://www.lyricsify.com".GetStringAsync();
    Debug.WriteLine($"lyricsifyResponse: {lyricsifyResponse}");
    // fails
    Assert.NotEmpty(lyricsifyResponse);
}

I checked the response object, it has status code 200 OK.
I checked redirects, which are enabled.

Any ideas? Did I miss something peculiar that lyricsify.com is doing? Do I need to configure anything in Flurl to get the expected web page as response?

Solution

Some websites expect the User-Agent header. Just pass some random value like this:

string lyricsifyResponse = await "https://www.lyricsify.com"
    .WithHeader("user-agent", "FlurlTest")
    .GetStringAsync();

Looks like Insomnia, Powershell and other tools set the user-agent header by default.