winhttp request only works when fiddler used as a proxy

I have never run into this before. I am trying to login to a website with the ultimate goal of downloading documents. in order to download documents I need a secure cookie which I get with winhttp. This website was complicated and involves many 302 redirects, but at last I get my final cookie. However the Url to download the document from is created on the fly by the server & it looks like it must match the cookie, so I must get one last page with winhttp & parse it for my URL. All my other request (12 of them) are good. This is a Get request & if I run it from Acesss i get 4000 bytes back instead of 17600. If i run fiddler as a proxy So I can inspect my request it works! but only thru fidler as a proxy. Turn off the proxy & it fails. It's not a total failure. If i alter the cookie i get 160 bytes & it tells me to "login again". the basics of this final request are :

Set WinHttpReq = CreateObject("WINHTTP.WinHTTPRequest.5.1")
    WinHttpReq.Open "GET", Trim(strRedirectURL), False            'get request
    WinHttpReq.Option(6) = False           'lets 302 codes come back. we then handle
    WinHttpReq.setRequestHeader "Host", "servicing1.foragentsonly.com"
    WinHttpReq.setRequestHeader "Connection", "keep-alive"
    WinHttpReq.setRequestHeader "Upgrade-Insecure-Requests", "1"
    WinHttpReq.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36"
    WinHttpReq.setRequestHeader "Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
    WinHttpReq.setRequestHeader "Sec-Fetch-Site", "same-site"
    WinHttpReq.setRequestHeader "Sec-Fetch-Mode", "navigate"
    WinHttpReq.setRequestHeader "Sec-Fetch-User", "?1"
    WinHttpReq.setRequestHeader "Sec-Fetch-Dest", "document"
    WinHttpReq.setRequestHeader "Referer", "https://www.foraonly.com/managepolicies/policyactivity/processeddateresults/newbusinessprocesseddate/"  
    WinHttpReq.setRequestHeader "Accept-Encoding", "gzip, deflate, br"
    WinHttpReq.setRequestHeader "Accept-Language", "en-US,en;q=0.9"
    WinHttpReq.setRequestHeader "Cookie", strStepTwelveCookie
    WinHttpReq.Send

I send this request after a 302 redirect. Unfortunately since fiddler works when I put it into proxy mode I cannot examine the request for difference. Looking at them they seem the same, which they should since it "worked". Turn off proxy, code fails, but fiddler doesn't capture my request so I cant compare. I've also tried without the Set line of code but I found my cookies being sent in duplicate? Any ideas on what I should look at would be appreciated. Thank you.

Edit: When I run this process thru Chrome Dev Tools, the pop up web page that I want, dev tools tells me its 4030 bytes. So possibly my winthhp request isnt failing. I see a few js functions on this popup webpage & I think one of them loads the page with content. If i'm right I dont know how to emulate this behavior with winhttp.

Solution

In your request you add the header

Accept-Encoding: gzip, deflate, br

This means that the server is allowed to send the reposne compressed using gzip or deflate(zip).

From what I have read about WINHTTP.WinHTTPRequest.5.1 it does not support automatic decompression of compressed responses. Therefore if you don't use Fiddler you see the compressed response.

But when you use Fiddler as proxy the requests is decompressed by Fiddler and then forwarded to your WINHTTP.WinHTTPRequest.5.1. In this case your scripts is getting the response body decompressed by Fiddler which is of course larger and contains the data you expect.