I am making a SOAP request to Salesforce Marketing Cloud API to retrieve Data Extension object records in batches of 2500 from an object who has 1,000,000 records total. Unfortunately the SDK is not an option due to an existing bug with respect to DE name lengths. Approximately halfway through, after running fine, I receive a 500 response and subsequent fail. From what I've read, it's due to throttling:
IN GENERAL, What is a good way to handle rate limiting within API requests? Do you add sleep timers to give more time in-between subsequent calls. Or would you catch the 500 response (or whatever error response you're receiving) and then add a sleep timer from there?
It's my first time running into this, so I appreciate any guidance / resources and am happy to test from there.
First and foremost, do not let your app make direct SOAP calls. Rather, write a utility library, and have the app always go through that.
Consult the documentation to learn the relevant rate limit policies. You need to know how many requests per hour would be acceptable.
Your library should maintain a global in-memory data structure which remembers recent transaction times and counts. That is, you should know how busy you've been lately, and how close to the edge of the rate limit you are skating. Consider sleeping a bit if you're getting close. Definitely log everything, with timestamps.
In the URL you cited, pay careful attention to the 2nd timestamp:
<wsu:Created>2017-02-01T19:07:24Z</wsu:Created>
<wsu:Expires>2017-02-01T19:12:24Z</wsu:Expires>
When your library eventually gets an error response, it should use that 2nd (future!) timestamp to intelligently choose how long to sleep. It is telling us when the ban will be lifted -- attempts before then will just be wasted.
Sleep slightly longer than needed. Your goal should be to always remain slightly below the limit.
Write unit / integration tests for this. The overall system is large and complex. You need to be confident you understand it and can predict its behavior. Sometimes there will be server-side changes, and you don't want those to create hard-to-diagnose Heisenbugs you must track down.
You need tests that deliberately trigger server-side rate limiting and then successfully recover. You also need tests that demonstrate high transaction rate which remains a little below the limit.