I am using quickfix python and connecting to an exchange via TCP connection to receive order book market data updates from them.
I am receiving market data incremental refresh messages (MsgType=X) at quite a high frequency from the exchange server. Originally there will be no difference between the receiving time and the time shown in field 52 (sendingTime).
From the FIX logs, the difference between receiving time and sendingTime gradually increases until it becomes a 1 hour difference as shown here : 20241112-21:00:01.000000000 : 8=FIX.4.452=20241112-19:51:16.421
What are possible reasons for this mismatch between the sendingTime recorded in the FIX logs and the actual time i received this message
QuickFIX handles messages on a single thread -- it won't start processing message B until it finishes processing message A.
Therefore, it is very important your QF callback implementations return quickly, or else you will start to see lag after handling a high volume of messages.
And I suspect this is exactly what is happening to you. You have logic in your onMessage(MarketDataIncrementalRefresh) (or whatever the Python version of that is) that is taking too long to complete, and it's causing a backup.
(One thing that does surprise me is that your app hasn't crashed out long before you hit the 1-hour lag. The default MaxLatency configuration is 120 seconds. Have you upped that value to a very large number?)
One solution:
Have your onMessage callback push incoming X messages to a queue, and have a separate thread consume these messages off that queue and do the processing that you need.
And of course, figure out way to speed up your processing logic.