azureazureservicebusazure-servicebus-topicsdead-letterazure-servicebus-subscriptions

Resubmitting messages from deadletter on Azure service bus on Topic queue with partitioning and duplicate detection enabled


I am trying to resubmit messages from the deadletter queue on a Azure service bus that is configured with a topic queue that has partitioning and duplicate detection enabled.

When I try to add one or more messages I get this error

System.Private.CoreLib: Exception while executing function: Functions.PartstatusResubmitFunction. Azure.Messaging.ServiceBus: Message to a partitioned entity with duplicate detection enabled must have either PartitionKey or MessageId set TrackingId:3dd6017e-c7ea-408e-b167-b5c78cfef8ed_G19,

I tried to give messageId and PartitionKey a new value on but it keeps giving me this error.

I set the MessageId and PartitionKey like this

ServiceBusMessage resubmitMessage = new ServiceBusMessage(dlqMessage);
Random random = new Random();
int randomNumber = random.Next(1, 1000);
resubmitMessage.MessageId = Guid.NewGuid().ToString();
resubmitMessage.PartitionKey = randomNumber.ToString();

This is done from an Azure function, where I use ServiceBusOutput to resubmit the message

[Function(nameof(PartstatusResubmitFunction))]
[ServiceBusOutput("partstatus", EntityType = 
ServiceBusEntityType.Topic, Connection = "connectionString")]
public async Task<List<ServiceBusMessage>> 
RunAsync([TimerTrigger("%CronTab%")] TimerInfo myTimer) 
{
if (myTimer.ScheduleStatus is not null)
{
    _logger.LogInformation($"Next timer schedule at: 
{myTimer.ScheduleStatus.Next}");
}

var props = await administrationClient.GetSubscriptionRuntimePropertiesAsync("partstatus", 
subscriberName.Replace("/$DeadLetterQueue", ""));
activeMessageCount = props.Value.ActiveMessageCount;
List<ServiceBusMessage> result = new List<ServiceBusMessage>();

if (activeMessageCount < 10000)
{
    var deadletterCount = await 
ProcessDeadLetterMessagesAsync($"topic: partstatus -> subscriber: 
{subscriberName}", 10, result);

    while (deadletterCount > 0)
    {
        deadletterCount = await 
ProcessDeadLetterMessagesAsync($"topic: partstatus -> subscriber: 
{subscriberName}", 10, result);
    }
}
else
{
    _logger.LogInformation("Not resubmitting messages from 
deadletter, ActiveMessageCount: " + activeMessageCount);
}

return result;

}

Thanks


Solution

  • I could resubmit and process the messages from the dead-letter topic of the Azure service bus using below code Azure function.

    Code Snippet:

        var connectionString = "<Service_Bus_connection_string>";
        var topicName = "<Topic_Name>";
        var subscriberName = "<subscription_name>";
        var fetchCount = 10;
        var client = new ServiceBusClient(connectionString);
        var sender = client.CreateSender(topicName);
        var dlqReceiver = client.CreateReceiver(topicName, subscriberName, new ServiceBusReceiverOptions
            {
                SubQueue = SubQueue.DeadLetter,
                ReceiveMode = ServiceBusReceiveMode.PeekLock
            });
    
        var wait = new TimeSpan(0, 0, 10);
    
        log.LogInformation($"fetching messages ({wait.TotalSeconds} seconds retrieval timeout)");
        log.LogInformation($"topic: {topicName} -> subscriber: {subscriberName}");
    
        IReadOnlyList<ServiceBusReceivedMessage> dlqMessages = await dlqReceiver.ReceiveMessagesAsync(fetchCount, wait);
    
        log.LogInformation($"dl-count: {dlqMessages.Count}");
    
            int i = 1;
            foreach (var dlqMessage in dlqMessages)
            {
                log.LogInformation($"start processing message {i}");
                log.LogInformation($"dl-message-dead-letter-message-id: {dlqMessage.MessageId}");
                log.LogInformation($"dl-message-dead-letter-reason: {dlqMessage.DeadLetterReason}");
                log.LogInformation($"dl-message-dead-letter-error-description: {dlqMessage.DeadLetterErrorDescription}");
    
                ServiceBusMessage resubmittableMessage = new ServiceBusMessage(dlqMessage);
    
                await sender.SendMessageAsync(resubmittableMessage);
    
                await dlqReceiver.CompleteMessageAsync(dlqMessage);
                log.LogInformation(dlqMessage.ToString());
                log.LogInformation($"finished processing message {i}: "+dlqMessage.Body.ToString());
                log.LogInformation("--------------------------------------------------------------------------------------");
                i++;
            }
    
             await dlqReceiver.CloseAsync();
            log.LogInformation($"finished");
    

    Console output:

    enter image description here

    Processing multiple Dead-letter messages:

    Functions:
    
            Function1: serviceBusTrigger
    
    For detailed output, run func with --verbose flag.
    [2024-03-27T09:32:11.187Z] Host lock lease acquired by instance ID '000000000000000000000000F72731CC'.
    [2024-03-27T09:32:21.404Z] Executing 'Function1' (Reason='(null)', Id=6fe5f19c-06ff-42a1-ab4b-1d9ac4c06efa)
    [2024-03-27T09:32:21.410Z] Trigger Details: MessageId: 30df9f87837f4aedb61ef4e262056801, SequenceNumber: 13, DeliveryCount: 1, EnqueuedTimeUtc: 2024-03-27T09:32:20.5520000+00:00, LockedUntilUtc: 2024-03-27T09:33:20.5520000+00:00, SessionId: (null)
    [2024-03-27T09:32:21.467Z] C# ServiceBus topic trigger function processed message: {MessageId:30df9f87837f4aedb61ef4e262056801}
    [2024-03-27T09:32:21.477Z] topic: topic1 -> subscriber: mysubscription
    [2024-03-27T09:32:25.834Z] dl-count: 2
    [2024-03-27T09:32:25.837Z] start processing message 1
    [2024-03-27T09:32:25.842Z] dl-message-id: 6373663f877e4431b9e57f29903b4492
    [2024-03-27T09:32:25.844Z] dl-messagereason: TTLExpiredException
    [2024-03-27T09:32:25.846Z] dl-message-description: The message expired and was dead lettered.
    [2024-03-27T09:32:28.037Z] finished processing message 1: Hello Pravu
    [2024-03-27T09:32:28.040Z] --------------------------------------------------------------------------------------
    [2024-03-27T09:32:28.043Z] start processing message 2
    [2024-03-27T09:32:28.046Z] dl-message-id: e9187857a5c442dbb9d20ed4bc37f50d
    [2024-03-27T09:32:28.049Z] dl-messagereason: TTLExpiredException
    [2024-03-27T09:32:28.052Z] dl-message-description: The message expired and was dead lettered.
    [2024-03-27T09:32:28.727Z] finished processing message 2: Hello
    [2024-03-27T09:32:28.731Z] --------------------------------------------------------------------------------------
    [2024-03-27T09:32:29.109Z] finished
    [2024-03-27T09:32:29.189Z] Executed 'Function1' (Succeeded, Id=6fe5f19c-06ff-42a1-ab4b-1d9ac4c06efa, Duration=7929ms)