Let's chat about ordering. It's one of my favorite topics, and something I've blogged about extensively before. Previously ordered processing in ...
For further actions, you may consider blocking this person and/or reporting abuse
This looks compelling when processing is successful, but what happens if a message in the session fails processing, say message #2? Will they still receive treatment and be discharged even though room assignment fails?
It will retry the message according to whatever retry policies you've set on the queue. So there's definitely a world you could see:
Eventually I expect the message would "deadletter" and processing on the session would continue. Here's the key line in the sessions doc:
We have done custom logic to handle this. We moved the message to a SQL table after the retries. If a new message arrives with same session available in SQL table, then the message will be moved to SQL table without processing. There was separate timer function to notify / process / cleanup the data in SQL table.
If that is the case, to ensure proper business logic as described in this scenario, I would not rely on message ordering.
Using only Azure Functions it would make more sense to me to use Durable Function(s) to orchestrate the business logic, probably using the external events and/or monitoring patterns.
Even with durable you’re going to have at-least-once guarantees. You have more control over some of the retries for sure, but a scenario of “message 1 is poisoned. It won’t ever successfully process” is going to be a scenario your app will need to come to terms with regardless if functions or not. Do you not ever process 2-4? If you tried to resubmit message 1 it would go to back of queue anyway, so how does your app get to it? You’re not wrong that durable may provide a bit more control here but some of the problems if you want your app to be “once and always once” successful delivery just aren’t very feasible in a distributed cloud world.
This is a pretty interesting approach. It puts me very much in mind of an Actor system. If this were to be coupled with durable functions to hold the state of the actor then I think you'd end up pretty close.
Great article. This solution fits my needs perfectly...almost. In my scenario it would be ideal if I could grab a whole batch of messages for a given session in a single Function invocation. Do you think we'll ever have a function trigger that supports this or is Event Hubs the way to go here? Hub triggers have the advantage (for me) of batching, but the disadvantage that partition keys are more "static". My scenario is closer to yours with Patient ID, where it would be nice if this partitioning were completely dynamic like how Sessions do it. So I'm a little torn on which messaging service to use.
Hi Jeff, We implemented this and are using the latest pre-release version of the trigger. We're still getting overlapping invocations for the same session ids. In our logs we write the session id and the lock period, but we get invocations for the same session concurrently. Any ideas?
We figured it out. Our function wasn't returning a Task.
HI Jeff, Just signed up on dev.to to say thanks. Have been struggling to find a solution approach for supporting multiple queues (Orders landing at various stores, need to be in sequence on a per store basis) was thinking of one queue per store and one function per queue, which would have made an unwieldy solution !
Really appreciate your post on using session with queue.
Cheers !
Is there a way to toggle revive modes from within the azure function? From my own experimentation, it seems that RecieveAndDelete is faster than PeekAndLock. For my use-case, I am okay with removing the message from the queue. I know there is an autocomplete option, but I'd like to experiment with receive modes if possible.
Thank this axticule is really hepfull, if in this implementation i want to set as complete to take out of thr queue or set as Defer for the cases that i had an extra business logic valdiation that i need to do that. how can i do this? since that i do not have a QueueClient to do this
Hi Jeff, thanks for the article. I want to activate sessions for a topic and its subscription. Is it enough to just set the flag requiresSession on the subscription or do I have to set something on the topic as well? Unlike for the queue, I can't find a setting "Enable sessions"
Jeff,
It seems that there have been some lingering issues with regards to how the ServiceBusTrigger connects to Service Bus and generating excessive requests (and thus costs).
github.com/Azure/Azure-Functions/i...
github.com/Azure/azure-webjobs-sdk...
Wondering if you have any observations with regards to the issues which were identified late last year.
Hi Jeff, great article. thanks a lot for writing this.
When Sessions are enabled, will each concurrent process (when maxConcurrentRequests >1 ) in the single Azure Function instance acquires lock for different sessions ?
Is there a limit to the number of sessions processed concurrently?
I.e. Functions will scale out, but will Service Bus broker keep up?
Suppose it depends on the SKU of service bus. You can modify the SessionHandlerOptions in the
host.json, so you can concurrently process multiple sessions on a single instance.great article. thanks a lot for writing this.
could you please share your thoughts for processing in-order for cosmosdb change feed trigger? From the docs I read, it's not clear to me how it works. We are planning to use cosmos for achieving atomicity between making an entry in db and raising an event. There are cases where I need to maintain order when processing the change feed. when azure functions are scaled out to multiple instances do they process change feed of a specific partition key in parallel? or even a single instance of azure function process feed messages in parallel?
Hi Jeff, thanks for the great article and example. One question I had. If you are using topics and subscriptions, is the session lock at the subscription level?
For example if you had two subscriptions (session enabled) with the enforce ordering on the topic. I assume the 2 subscriptions would both get copies of the messages (in order), if you had two azure functions each listening to a respective subscription, the order of those triggering functions maybe in any order?
I think the order of the messages themselves would be in-order delivered to each function. But the functions could possibly execute at the same or different times?
Great article, going to implement this :)
Nice article!! I am looking for the same. But I just wanted to know how can I read specific queue messages by Session Id in Azure Function Service Bus Trigger.
Please suggest
Hi Jeff, excellent post, what happen if we have more than 32 messages for a specific session?
When a function places a lock on a session, it may make multiple requests to get many messages. You could have thousands of messages for the same session, the function app will process one at a time and then go back and ask service bus for the next one. With sessions you could even send a message to a session 1 year later, a function app will wake up, lock the session, and process that message. So really no limit to how many or how frequently you can push messages to a session.
In the case of sessions, because each message must be processed in order, you really get the concurrency from having multiple sessions processed at once on an instance, but to preserve ordering only one message from a session will be pulled at a time (if that makes sense)
docs.microsoft.com/en-us/azure/ser...
Hi Jeff,
i have put my comments in below link.
github.com/jeffhollan/functions-cs...
Please look into it
Previously, I had to use Durable Functions with Monitors, Table Storage and Redis to achieve this. This makes it simple, easier and more efficient.
Hi Jeff,
Do the sessions work with Azure function with consumption plan? Thanks!
Regards,
Dmitrii
Yes - will work in all environments
I am routing Device Twin Changed events from the IoT Hub to the service bus queue. Is There a way I can use the DeviceId as a SessionId?
Hi @jeffhollan , Thanks so much for writing this detailed article.
I have a question on a similar line with Silvan. It was mentioned in the microsoft article that -Service Bus queues and topics used as IoT Hub endpoints must not have Sessions or Duplicate Detection enabled. If either of those options are enabled, the endpoint appears as Unreachable in the Azure portal. (learn.microsoft.com/en-us/azure/io...).
But now if I want to ensure ordering on these messages from iotHub, how can I do that?
Appreciate your help and time!!