Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Azure Functions: Queue Trigger is expecting Base-64 messages and doesn't process them correctly

I have this Queue Trigger. The expected is when I insert a message in the Queue, the trigger must fire and process the dequeued message.

    [FunctionName("NewPayrollQueueTrigger")]
    public async static void Run([QueueTrigger("myqueue", Connection = 
    "AzureWebJobsStorage")]string myQueueItem,
        [DurableClient] IDurableOrchestrationClient starter,
        ILogger log)
    {
        log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");

        await starter.StartNewAsync("NewPayrollOrchestrator", input: myQueueItem);

    }

The trigger is being activated normally, but this weird behavior is happening. The function apparently expects that the message is encoded in Base-64.

Exception binding parameter 'myQueueItem' <--- The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.

I'm sending messages to the queue using this method from the Azure Queue library v.12 from Azure.Storage.Queues and found no overloads that encodes the message to Base-64. Note that _queue is a QueueClient instance.

 public async Task<Response<SendReceipt>> SendAsync(string message)
 {
        return await _queue.SendMessageAsync(message);
 }

So I tried to encode the message by myself...

 public async Task<Response<SendReceipt>> SendAsBase64Async(string message)
 {
      byte[] buffer = Encoding.Unicode.GetBytes(message);
      string msg = Convert.ToBase64String(buffer);

      return await _queue.SendMessageAsync(msg);

 }

... and it doesn't work either. Here's my code passing by that part but throwing error further on, indicating that it could get the message but it was not decoded correctly, since it was a filename of an existing blob in a storage:

Decode error

The only way to get this working is if I manually send a message to the queue using the Azure Storage Explorerchoosing for encode the message via UI.

like image 514
Ramon Dias Avatar asked Jul 21 '20 21:07

Ramon Dias


People also ask

How do I use queuetrigger with Azure web jobs storage?

The "AzureWebJobsStorage" app setting. The QueueTrigger annotation gives you access to the queue that triggers the function. The following example makes the queue message available to the function via the message parameter. Declares the parameter name in the function signature.

What happens when a queue trigger fails?

When a queue trigger function fails, Azure Functions retries the function up to five times for a given queue message, including the first try. If all five attempts fail, the functions runtime adds a message to a queue named <originalqueuename>-poison.

How to create a service bus queue in Azure Functions?

As you can see, The template offers a wide variety of triggers such as HTTP, RabbitMQ, or SignalR. Select Service Bus Queue trigger and on the right side enter a name for the connection string and the queue name. Click on Create and your first Azure Function gets created.

How do I get Poison messages from Azure Functions?

Poison messages When a queue trigger function fails, Azure Functions retries the function up to five times for a given queue message, including the first try. If all five attempts fail, the functions runtime adds a message to a queue named <originalqueuename>-poison.


3 Answers

Use Azure.Storage.Queues nuget package and use the following code to convert string to Base 64 encode. You need to encode using Encoding.UTF8.GetBytes (plainText).

await queueClient.SendMessageAsync(Base64Encode(serializedCommand), cancellationToken);

private static string Base64Encode(string plainText)
{
    var plainTextBytes = System.Text.Encoding.UTF8.GetBytes(plainText);
    return System.Convert.ToBase64String(plainTextBytes);
}

For more details, you could refer to this issue.

like image 41
Joey Cai Avatar answered Oct 22 '22 13:10

Joey Cai


As an alternative, you can take advantage of the built in functionality for converting the message to Base64 by creating the client with an QueueClientOptions with MessageEncoding set to QueueMessageEncoding.Base64.

For example:

_queue = new QueueClient(connectionString, queueName, new QueueClientOptions
{
    MessageEncoding = QueueMessageEncoding.Base64
});

...

var message = "some message";
await _queue.SendMessageAsync(message); // Will be converted as Base64.
like image 85
Johan Classon Avatar answered Oct 22 '22 13:10

Johan Classon


If you're using version 5.0.0 or higher of Microsoft.Azure.WebJobs.Extensions.Storage, you can set the message encoding in host.json:

{
    "version": "2.0",
    "extensions": {
        "queues": {
            "messageEncoding": "none"
        }
    }
}

https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue#hostjson-settings

like image 5
Daniel Sklenitzka Avatar answered Oct 22 '22 13:10

Daniel Sklenitzka