I just started with a company which has a home grown cloud solution. They have built their central queuing mechanism around SQL Server. When talking to the technical director, he tells me that they have tried MSMQs in the past, but ran into issues with high volume queues getting corrupted. I have been doing a search on the internet, but cannot find anything about this issue.
Has anyone heard of such a thing? Are there any good articles on this?
Also, they are not using WCF for this purpose. Would the transactional nature of WCF solve these corruption issues?
No, MSMQ does not get corrupted.
What Hugh is referring to is the memory mapping of storage files into virtual memory and the fragmentation of such.
MSMQ works in 4MB blocks, regardless of the message size. Send a 5kb message - get a 4MB block. Send another 5kb message - use the same block. Eventually the block fills up and MSMQ starts a fresh one. When all the messages in a block are read and consumed (as opposed to just peeked), the file is deleted. MSMQ waits several hours before deleting, just in case a new message arrives and needs a home - no point creating a new file when there's one hanging around empty.
If the messages are of the same size, the blocks are easily filled and emptied as they arrive and depart.
If the message are of variable size, and are not read FIFO, then holes can start appearing. For example, a 5kb message is read and a 4 kb message arrives, leaving 1kb of unusable space. Over time, the amount of allocated memory will deviate from the actual volume of messages due to accumulated dead space.
This is all normal. There is no defrag for MSMQ.
Notes