I have a quickfixJ Java application that have been running for months. A weird scenario happened recently which is that an order execution was logged in FIX message logs but was not received by the Java application (quickfixJ MessageCracker). No rejection messages (35=3) appear in the messages logs. What can be the cause of this issue? and how to prevent it?
The transactions messages were logged in the FIX messages logs at time: 16:08:45.677 In the event log during the same min, no logs, but after half a minute we have:
20230515-12:43:35: Sent SequenceReset TO: 7
20230515-16:09:31: Sent test request TEST
20230515-16:09:58: Disconnecting: Timed out waiting for heartbeat
20230515-16:09:58: java.lang.reflect.UndeclaredThrowableException
java.lang.reflect.UndeclaredThrowableException at
com.sun.proxy.$Proxy183.onConnect(Unknown Source) at
quickfix.Session.setResponder(Session.java:567)
Example of the message log that did not reach Java:
8=FIX.4.49=37835=849=XXXX56=VVVV34=93852=20230515-16:08:45.69337=17070005240=2526=BI-710202-I11=41F5517A-0563-6F62-1CB0C34522E02B8B453=1448=307802=1523=VVVV382=1375=307337=VVVV17=1707000_29009=792443150=F39=21=L1068955=STAC48=CI000000035222=454=238=9040=244=19559=6432=2023051532=9031=195151=014=906=060=20230515-16:08:45.537381=17550.00111=010=056
Additional info: To note that we were using Windows server and we recently moved to Unix server.
Another example that occurred today:
Fix message received and logged in FIX messages log at 20230516-10:48:46.816:
8=FIX.4.49=39735=849=vvv56=xxx34=137852=20230516-10:48:46.81637=17070255240=1526=CI-710206-I11=6A1FAA42-0453-1001-0FAA9A63AD1F0620453=1448=307802=1523=xxx17=1707025_1150=039=01=L1068955=NSBC48=CI000000241622=454=138=140=244=338059=0151=114=06=060=20230516-10:48:46.660111=158=Entr?e de l'ordre: Ordre ? cours limit?, Achat 1 actions NSBC ? 3?380,00, ordre valable jour.10=111
During this time, the event log shows the following:
20230516-08:57:40: Received ResendRequest FROM: 2 TO: infinity
20230516-08:57:40: Resending message: 2
20230516-08:57:40: Resending message: 3
20230516-08:57:40: Resending message: 4
20230516-08:57:40: Resending message: 5
20230516-08:57:40: Sent SequenceReset TO: 7
20230516-10:40:29: Sent test request TEST
20230516-10:44:52: Sent test request TEST
20230516-10:46:55: Sent test request TEST
20230516-10:48:03: Sent test request TEST
20230516-10:49:12: Sent test request TEST
20230516-10:49:26: Reject sent for message 1340: SendingTime accuracy problem:52
20230516-10:49:26: Reject sent for message 1341: SendingTime accuracy problem:52
20230516-10:49:26: Reject sent for message 1342: SendingTime accuracy problem:52
20230516-10:49:26: Reject sent for message 1343: SendingTime accuracy problem:52
20230516-10:49:26: Reject sent for message 1344: SendingTime accuracy problem:52
20230516-10:49:26: Reject sent for message 1345: SendingTime accuracy problem:52
20230516-10:49:26: Reject sent for message 1346: SendingTime accuracy problem:52
20230516-10:49:26: Reject sent for message 1347: SendingTime accuracy problem:52
20230516-10:49:30: Disconnecting: Timed out waiting for logout response
And the message was not received from Java code side.
Initial message sent for order:
8=FIX.4.49=19535=D34=44749=xxx52=20230515-16:08:32.50456=vvv1=L1068911=420F77FC-C7EB-972F-A9C15707279A78B438=2040=244=90054=255=FTSC59=660=20230515-16:08:32.504432=20230531526=BI-710201-I10=115
i also saw some other error messages in the event log like:
20230516-10:49:31: quickfix.SessionException Logon state is not
valid for message (MsgType=AE)
20230516-10:49:31: Already disconnected: Verifying message failed: quickfix.SessionException: Logon state is not valid for message (MsgType=AE) 20230516-10:49:31: quickfix.SessionException Logon state is not valid for message (MsgType=W) 20230516-10:49:31: Already disconnected: Verifying message failed: quickfix.SessionException: Logon state is not valid for message (MsgType=W)
I then tried to add CheckLatency N to stop disconnecting in case of time differences, but i am still facing issues which shows that the order market reply has a time prior to the order booking request (by seconds) this is resulting in delays in receiving the message in Java listener although that fix messages are well logged in the FIX messages.
The issue was solved by removing the delay that is present in the message processing, as when Java takes some time to process the first message, then the second message can be lost although logged in FIX messages.