Can you detail more on the scenarios you're thinking of for it, and how you'd like it to work? The new extensibility model we're opening up would allow you to write one yourself if you wanted to. However, if this is something that many people would find useful, it would make sense for us to bake into our core ServiceBus assembly.

It's not clear to me whether this would need to be a completely new trigger binding, or whether on the trigger/receive side of things it couldn't just be an additional mode on the existing ServiceBusTrigger binding. Behind the scenes, it polls for and accepts sessions, and invokes your job function for session messages.

As far as sending session based messages, I think our ServiceBus binding (e.g. used when sending a BrokeredMessage via out param binding) would work as is? You just set your session ID on the message before sending.

I'm going to use it for handling the messages of one session in a specific sequence and preferably multiple sessions at once. Since it's currently not possible, I'm going to use this article to process multiple sessions per webjob.
How should it work? Good question... I think it would be nice to have a method something like this:

public static void ProcessQueueMessages([ServiceBusTrigger(queueName:"inputqueue", useSessions: true)] List<string> messages) { ... }

Or whatever type of object you expect. Also you would need to specify somewhere in the config the timeout between sessions; after how many (milli)seconds will you assume the session is finished?

And you are correct, for sending messages you just need to set something in the SessionId.

I'd love to see this feature as well. I need my inbound messages to be in a guaranteed delivery order, and it seems using the Service Bus Session is the way to do this. However it is not at all clear how I would implement this in a WebJob.

@mathewc

Not having this is a blocker for me currently. We're using queues + the webjobs SDK as a very lightweight, scalable reactive work item list - messages come on, and processing those messages leads to 0 .. M new messages going back onto the queue. When we need to have guaranteed sequential processing - almost simulating an actor-style model - service bus sessions is the only option. This is crucial if we have some critical region of data processing and don't want e.g. transactions on SQL database (or can't e.g. Azure blobs).

Having an easy way to target messages on a specific session and have them work transparently within webjobs would be a huge benefit and would open a ton of possibilities. So on the input side I would love to see it "just work": -

• Web job trigger has an extra argument to indicate if you want to bind to sessions or just plain messages
• A session is bound to a specific web job process + thread instance and stays with it until the session dries up
• Optionally an extra input argument for the name of the session that has been bound to the function call
• On the output, it would be great if you could specify the session name on the ServiceBus attribute somehow, but this isn't as much of an issue as there's already a way to do this via BrokeredMessage directly as you say.

I need this feature as well. We have a WebAPI call that's taking longer and longer as we add more work to each call. When it started to timeout in Azure, we decided to move the process into a WebJob so that clients can initiate the task, then check back to get progress. The clients in this case are browsers making AJAX calls. If we can't correlate the requests with the responses, they can't check on progress or completion of the WebJob. We started looking at Azure Storage Queues, but found that there's no way for the original caller to only receive responses intended for them. We can't create a response queue for each client, because each client is a browser. We saw that Service Bus Queues and Topics allow us to use the SessionId to correlate requests and responses. The client (browser) can filter the response queue to only see responses intended for them. It sounded like a perfect solution. After wrestling with configuration we got the JobHost.RunAndBlock() to almost run, but we get the following error:

It is not possible for an entity that requires sessions to create a non-sessionful message receiver.

We found this documentation at http://www.cloudcasts.net/devguide/Default.aspx?id=13029

If sessions are required on a queue it is not possible to use the QueueClient Receive method to dequeue messages. Attempting to do so will result in.

InvalidOperationException

It is not possible for an entity that requires sessions to create a non-sessionful message receiver.

We assume the WebJobs SDK is using the QueueClient Receive method. We're going to try to work around this by having the request queue send a Guid as part of the payload and not require Sessions on the request queue. We'll require sessions on the response queue and see if we can use the Guid in the payload as the SessionId for the response. Will let you know if that works.

One thing I've noticed since my OP is that you can use [<Singleton>] on e.g. a Storage Queue and use the SingletonScope to achieve virtually the same thing (as far as I can see) as Service Bus sessions. Can anyone confirm that?

FYI, the work-around mentioned above worked perfectly:
"having the request queue send a Guid as part of the payload and not requiring Sessions on the request queue. We'll require sessions on the response queue and use the Guid in the payload as the SessionId for the response."

I would really like to see this implemented as well. I don't use the WebJobs SDK directly that much, but if I understand it correctly this would allow for Azure Functions written in for example JavaScript to handle message sessions (something that's not supported in the Node.js Azure SDK since it uses the REST interface of Azure Service Bus).

Was there any traction on this?

+1
Our requirement is FIFO by tenant in a multi-tenant system. In this case data imports via upsert

@amit-kumar2 -- is this the issue that you brought up to me yesterday? If so, would you want to weigh in with your scenario here so we have it captured?

HI,

i am using below code with Servicebus and Topic-Subscription with session enabled

public static void ProcessTopicMessage([ServiceBusTrigger("%ServicebusTopic%", "%ServicebusSubscription%")] BrokeredMessage message, TextWriter logger)
{
}

but the problem i face with continuously running web job is host.RunAndBlock(); throws below error so how to resolve this issue? Azure web job does not support Session enabled topic-subscription?

An unhandled exception of type 'System.InvalidOperationException' occurred in mscorlib.dll

Additional information: It is not possible for an entity that requires sessions to create a non-sessionful message receiver. TrackingId:074fbc74-b5e9-4fc2-886b-ed2f4e324304_G5_B15, SystemTracker:az-g3ms-servicebus:Topic:az-rsd-dev-topic|az-rsd-dev-subscription, Timestamp:5/11/2018 5:16:14 AM

Hi @appalaraju were you able to resolve your issue?. In my case I'm not getting any exceptions, but the web job is not being fired either. If I disable sessions in my queue the web job starts working again.

As far as we have been able to determine, we simply have to accept that the queue will throw exceptions on occasion with no discern-able reason so we just have to handle and ignore it. As far as not working when sessions are enabled, we have not encountered that. Have you tried turning on diagnostics in the portal?

Related to background errors coming from MessageReceivers - we just made some improvements in that area: Azure/azure-webjobs-sdk#1703

Hi @raulbojalil,MikeYeager

Have you tried with Topic + Session enabled Subscription?

in my case, i have created a web job and used the code(ProcessTopicMessage) which i specified above with Topic and session enabled subscription and ran it locally( in my machine).

i am facing the issue which i specified above.

any idea on when can Azure web job works properly with Topic + Session enabled Subscription?

Warning: This horse is dead 🐴!

You need to process your messages in FIFO style? Better use some other technology and stop waiting for Microsoft to support their own technology stack!

If Microsoft cannot prioritize this, would anybody else be interested in attempting to put together a PR for this? I am not sure I can do it alone, but I'm happy to take a stab at it.

It'd be good to have this feature :(

So it's been over 2 years since this issue was raised and still nothing on whether/when this will be addressed.

Please vote for it at https://feedback.azure.com/forums/355860-azure-functions/suggestions/13882992-support-for-session-enabled-azure-service-bus-queu

This is a limitation for us, any eta to implement this ?

@ridhoq If you want I join the community support for this

On October 18th 2018 a comment was added to https://feedback.azure.com/forums/355860-azure-functions/suggestions/13882992-support-for-session-enabled-azure-service-bus-queu to say the work has been planned... so there is hope!

Assigning this to sprint 45 for initial design and scoping

Design is ready for internal review. Will be reviewing and planning in Sprint 47. When finalized, we'll respond back here.

Design work completed. Moving this to sprint 48 to track the implementation

Pre-release nuget package is out. Change Microsoft.Azure.WebJobs.Extensions.ServiceBus nuget package in your VS project if you want to try to:
https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.ServiceBus/3.1.0-beta1

Using:
public static void Run([ServiceBusTrigger("core-test-queue1-sessions", Connection = "AzureWebJobsServiceBus", IsSessionsEnabled = true)]string myQueueItem, ClientEntity clientEntity, ILogger log)

IsSessionsEnabled = true - if you are pointed to a sessions enabled queue/subscription.
ClientEntity clientEntity - for management operations (like MessageReceiver for a regular queue/subscription).

Also you can specify new SessionHandlerOptions section in host.json:

{
    "version": "2.0",
    "extensions": {
        "serviceBus": {
            "SessionHandlerOptions":
             {
                "MaxAutoRenewDuration": "00:01:00",
                "MessageWaitTimeout": "00:05:00",
                "MaxConcurrentSessions": 16,
                "AutoComplete": true,
             }
        }
    }
}

I'll appreciate for any feedback.

Moving this to sprint 49 to allow more time for feedback before this work is merged.

Is this feature available in Azure functions?

Hi @alrod,

I am doing a proof of concept of using ServiceBus session feature in one of our Azure Functions. The Azure Function that I am trying to enable ServiceBus session has AutoComplete set to false and the function manually Complete, DeadLetter or Abandon depending on the outcome. When use IMessageSession Complete, DeadLetter or Abandon I exeperiece intermittent SessionLockLostException under load and evntually succeed. But if I set AutoComplete to true I don't see this error at all. This is the only issue we have faced so far blocking us using this feature. Are you able to shed some light this issue?

Thanks

Hi @alrod,

Question: Service Bus Explorer has a queue option "Enforce Message Ordering". What is this option for? Should it be set along with enabling sessions if we want to guarantee processing in the order received?

https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.ServiceBus/3.1.0-beta4 is out.

@tidusjar, @mpnow the issue for resolving %Variable% is fixed in beta4.
@rybczak, most likely the feature will be released in September

@alrod Still aiming for a release in this month?

Are async methods supported? Running a test today I published a single message with a session id and processed it using a method with the following signature:

public static async Task Run([ServiceBusTrigger("name", Connection = "connection", IsSessionsEnabled = true)]Message message, ILogger log)
{
   await doThingAsync();
}

As soon as the debugger hit the async call in the code, the Run method was invoked a second time with the same message (identical message id, session id, etc.) Both instances of the message being processed completed with no exceptions being thrown. I verified over several runs that this behavior was happening with only a single message in the queue.

:EDIT: Nevermind. I was debugging and the lock was expiring, causing the message to be grabbed again.

I'm testing this session capability and am getting frequent occurrences of the error "The session lock has expired on the MessageSession". Can anyone explain why this might be happening?

When I use these options:

      "SessionHandlerOptions": {
        "MaxAutoRenewDuration": "00:01:00",
        "MessageWaitTimeout": "00:00:10",
        "MaxConcurrentSessions": 1,
        "AutoComplete": true
      }

then when the error occurs, it stops all processing of my Azure function - the only recourse seems to be to restart the function. I'm assuming that's occurring because I have MaxConcurrentSessions set to 1, so when my one concurrent session goes into that error state, no further processing can occur.

When I remove those SessionHandlerOptions and just let everything be defaults, I still get the error, but processing is still able to continue. I'm assuming that's because while that function instance is dead in the error state, processing can continue using another instance. If I'm right, then that's only a temporary solution as I will still run out of instances once they all hit that error.

So I really need to figure out why the error is occurring in the first place and how to prevent it. Anyone have any ideas?
@alrod

https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.ServiceBus/3.1.0-beta4 is out.
@tidusjar, @mpnow the issue for resolving %Variable% is fixed in beta4.
@rybczak, most likely the feature will be released in September

@alrod Still aiming for a release in this month?

Sessions support is released in https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.ServiceBus/3.1.0

I'm testing this session capability and am getting frequent occurrences of the error "The session lock has expired on the MessageSession". Can anyone explain why this might be happening?

When I use these options:

      "SessionHandlerOptions": {
        "MaxAutoRenewDuration": "00:01:00",
        "MessageWaitTimeout": "00:00:10",
        "MaxConcurrentSessions": 1,
        "AutoComplete": true
      }

then when the error occurs, it stops all processing of my Azure function - the only recourse seems to be to restart the function. I'm assuming that's occurring because I have MaxConcurrentSessions set to 1, so when my one concurrent session goes into that error state, no further processing can occur.

When I remove those SessionHandlerOptions and just let everything be defaults, I still get the error, but processing is still able to continue. I'm assuming that's because while that function instance is dead in the error state, processing can continue using another instance. If I'm right, then that's only a temporary solution as I will still run out of instances once they all hit that error.

So I really need to figure out why the error is occurring in the first place and how to prevent it. Anyone have any ideas?
@alrod

@mpnow, Default lock session period is 30 secs (check messageSession.LockedUntilUtc). You can extend it using MaxAutoRenewDuration. You are getting SessionLockLostException if function execution time more then lock duration. Function runtime tries to complete the message with expired lock.

Regarding stopping processing if "MaxConcurrentSessions": 1. I reproduced it and created the issue: #26. Thanks for reporting.

Hi @alrod,

I am doing a proof of concept of using ServiceBus session feature in one of our Azure Functions. The Azure Function that I am trying to enable ServiceBus session has AutoComplete set to false and the function manually Complete, DeadLetter or Abandon depending on the outcome. When use IMessageSession Complete, DeadLetter or Abandon I exeperiece intermittent SessionLockLostException under load and evntually succeed. But if I set AutoComplete to true I don't see this error at all. This is the only issue we have faced so far blocking us using this feature. Are you able to shed some light this issue?

Thanks

@dinukajayamuni, please see #16 (comment).

I am getting SessionLockLostException using https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.ServiceBus/3.1.0 for both:

1. autoComplete = true

    //{
    //  "version": "2.0",
    //  "extensions": {
    //    "serviceBus": {
    //      "sessionHandlerOptions": {
    //        "maxAutoRenewDuration": "00:00:30",
    //        "autoComplete": true
    //      }
    //    } 
    //  }
    //}

    [FunctionName("Function1")]
        public static async Task Run([ServiceBusTrigger("xxx", IsSessionsEnabled = true)]Message message, IMessageSession messageSession, ILogger log)
        {
            //await Task.Delay(25000); // no SessionLockLostException
            await Task.Delay(50000); //SessionLockLostException is throw
        }
    }

2. autoComplete = false

    //{
    //  "version": "2.0",
    //  "extensions": {
    //    "serviceBus": {
    //      "sessionHandlerOptions": {
    //        "maxAutoRenewDuration": "00:00:30",
    //        "autoComplete": false
    //      }
    //    } 
    //  }
    //}

    [FunctionName("Function1")]
        public static async Task Run([ServiceBusTrigger("xxx", IsSessionsEnabled = true)]Message message, IMessageSession messageSession, ILogger log)
        {
            //await Task.Delay(25000); //no SessionLockLostException
            await Task.Delay(50000); //SessionLockLostException is throw
            await messageSession.CompleteAsync(message.SystemProperties.LockToken);
        }
    }

Hi @alrod ,

Thanks for the above info about the session lock timeout. I'm still having troubles, though. No matter what I set MaxAutoRenewDuration to in host.json (for example, I tried setting it to 00:09:00), I still get a lock error if my function runs for more than a minute or so. Also when I check messageSession.LockedUntilUtc, it always shows a value about one minute into the future. (I've tried setting MaxConcurrentSessions to 1 and to values greater than 1 but that doesn't change the behavior - still get lock errors regardless.)

I notice there's a messageSession.OperationTimeout that's always set to 00:01:00, could that have any bearing here? Also note that I'm running these tests locally, but that shouldn't matter, should it?

Hi @alrod,
Below is the sample , i am using for sending messages to topic-subscription.
may i know how to proceed with this problem?

static DeviceClient deviceClient;
var commandMessage = new Message("test");
commandMessage.Properties.Add("Component", "rs-dev"); // qa
commandMessage.Properties.Add("Channel", "cloud.temperature");
commandMessage.Properties.Add("Source", "Source");
await deviceClient.SendEventAsync(commandMessage);

i am using IOT Hub endpoints and MessageRouting to forward messages from device to required topic

here "Message" class belongs to Microsoft.Azure.Devices.Client so i am unable to keep "SessionId" for this "Message" class.

even i keep below statement to keep "SessionId" for message, session concept is not working after reaching this message to topic
commandMessage.Properties.Add("SessionId", "500");

could you please let me know how to apply "SessionId" to above scenario?

@appalaraju , I am not aware how to sent SessionId using Microsoft.Azure.Devices.Client. Try to ask here: https://github.com/Azure/azure-iot-sdk-csharp

Hi @alrod ,

Thanks for the above info about the session lock timeout. I'm still having troubles, though. No matter what I set MaxAutoRenewDuration to in host.json (for example, I tried setting it to 00:09:00), I still get a lock error if my function runs for more than a minute or so. Also when I check messageSession.LockedUntilUtc, it always shows a value about one minute into the future. (I've tried setting MaxConcurrentSessions to 1 and to values greater than 1 but that doesn't change the behavior - still get lock errors regardless.)

I notice there's a messageSession.OperationTimeout that's always set to 00:01:00, could that have any bearing here? Also note that I'm running these tests locally, but that shouldn't matter, should it?

@mpnow, can you please upload the project with repro so I can take a look.

ppalaraju , I am not aware how to sent SessionId using Microsoft.Azure.Devices.Client. Try to ask here: https://github.com/Azure/azure-iot-sdk-csharp

Thanks @alrod

Hi @alrod ,

Thanks for the above info about the session lock timeout. I'm still having troubles, though. No matter what I set MaxAutoRenewDuration to in host.json (for example, I tried setting it to 00:09:00), I still get a lock error if my function runs for more than a minute or so. Also when I check messageSession.LockedUntilUtc, it always shows a value about one minute into the future. (I've tried setting MaxConcurrentSessions to 1 and to values greater than 1 but that doesn't change the behavior - still get lock errors regardless.)

I notice there's a messageSession.OperationTimeout that's always set to 00:01:00, could that have any bearing here? Also note that I'm running these tests locally, but that shouldn't matter, should it?

@alrod I experience the same behavior. No matter maxAutoRenewDuration setting, operation timeout is always set to 1 min

@pzbyszynski, messageSession.OperaionTimeout is nothing to do with maxAutoRenewDuration. It's "Duration after which individual operations will timeout.".
Working sample, execution time 1 min:
Function1.zip

@alrod Your sample works indeed.

Nevertheless, in my production code I have my function set up in the following way:

  "extensions": {
    "serviceBus": {
      "messageHandlerOptions": {
        "maxConcurrentCalls": 4,
        "maxAutoRenewDuration": "00:10:00"
      },
      "sessionHandlerOptions": {
        //"maxConcurrentSessions": 4,
        "maxAutoRenewDuration": "00:10:00"
      }
    }
  }     

and I have still tons of Microsoft.Azure.ServiceBus.SessionLockLostException on functions that run for example 3 minutes.

Any input on that would be highly appreciated @alrod

@alrod
I'm working with a simple setup where I have AutoComplete = false and I complete the messages myself, which works as attended for the most part.
I regularly experience SessionLockLostExceptions though, even in the most basic setup where there is plenty of session-lock-time left. With a little digging I found out that my function instances quite often are injected with the wrong IMessageSession objects. I can easily tell, because the Msg.SessionId is different from the messageSession.SessionId.
It seems that the following scenario takes place:
Two messages in two different sessions are on the subscription that my function subscribes to. Then my function is triggered twice, almost simultaneously, and both sessions are correctly locked, but both function instances are injected with the same messageSession. Then when I try to complete a message with the wrong messageSession, it of course fails, but it surfaces as a SessionLockLostException, which is possibly what a lot of the others in this thread are encountering.

@alrod
I'm working with a simple setup where I have AutoComplete = false and I complete the messages myself, which works as attended for the most part.
I regularly experience SessionLockLostExceptions though, even in the most basic setup where there is plenty of session-lock-time left. With a little digging I found out that my function instances quite often are injected with the wrong IMessageSession objects. I can easily tell, because the Msg.SessionId is different from the messageSession.SessionId.
It seems that the following scenario takes place:
Two messages in two different sessions are on the subscription that my function subscribes to. Then my function is triggered twice, almost simultaneously, and both sessions are correctly locked, but both function instances are injected with the same messageSession. Then when I try to complete a message with the wrong messageSession, it of course fails, but it surfaces as a SessionLockLostException, which is possibly what a lot of the others in this thread are encountering.

I can confirm this. Any update on this issue?

@cgrevil, @buyckkdelaware. I will take a look on the issue during next sprint:
#23

The feature is already released.

Guys, I need some suggestions on implementation of concurrent sessions.
I have a scenario with patient_1,patient_2,patient_3 etc..
I have 5(1,2,3,4,5) messages for patient_1, 3(1,2,3) messages for patient_2, 4(1,2,3,4) messages for patient_3 etc..
I need to execute the patients in a parallel but my message-id should be in a sequential(one after another).
i.e,
Patient_1 executing message_id 1
Patient_2 executing message_id 1
Patient_3 executing message_id 1

if patient_2 completes first then it should pick the next message-id and process .....

They all should execute parallel and once they finish executing their respective message then it should pick up the next one and process accordingly.
Here my intention is patients should be executed in a concurrent manner(>1). But the messages should go in a sequential way(max concurrent should be 1)

Any comments or suggestions are welcome.
Thanks in advance :)

Hello, I am facing the issue with the Microsoft.Azure.ServiceBus.SessionLockLostException as well. From what I found even though you are increasing the MaxAutoRenewDuration it won't help. The thing you need to increase is the Subscription's LockDuration(by default 30 seconds), but the maximum allowed value is 5 minutes. When I try to increase it over that value I get the following exception:
Actual value was 00:07:00. TrackingId:19acead3-ae8e-4c0c-92dd-3d4cd61126b9_G13, SystemTracker:sometopic:Topic:somesubsciption, Timestamp:2020-06-30T08:56:09. Method <UpdateSubscription>b__0: retry 5 of 10. <11:55:52> Exception: The remote server returned an error: (400) Bad Request. The supplied lock time exceeds the allowed maximum of '5' minutes. To know more visit https://aka.ms/sbResourceMgrExceptions. Parameter name: LockDuration

I am using ServiceBusExplorer to manage my topics and subscriptions.

What you have to do is manually renew the session lock before it expires, so you don't get the same message from the queue with the new lock while you are still processing the first arrived one. But the IMessageSession.LockedUntilUtc property returns a value that is a bit larger from what I have set to the LockDuration. So for the moment I have to hardcore it and don't depend on the IMessageSession properties.

What you have to do is manually renew the session lock before it expires

for the moment I have to hardcore it

Can you provide more detail as to exactly what code you're executing to do the above? It would be much appreciated.

What you have to do is manually renew the session lock before it expires

for the moment I have to hardcore it

Can you provide more detail as to exactly what code you're executing to do the above? It would be much appreciated.

So in the handler I have registered with SubscriptionClient.RegisterSessionHandler(), I do the following:

` var timeout = 25 * 1000;
var manualResetEventSlim = new ManualResetEventSlim(false);

        var timer = new System.Threading.Timer(state =>
        {
            ((ManualResetEventSlim)state).Reset();
            Log.LogInformation("Renewing session lock manually!");
            session.RenewSessionLockAsync().GetAwaiter().GetResult();
            ((ManualResetEventSlim)state).Set();
        }, manualResetEventSlim, timeout, timeout);

        token.Register(() =>
        {
            manualResetEventSlim.Wait();
            timer.Dispose();
        });`

I dispose the timer also when I complete or sent the message to the deadletter queue. As the service I am working on needs to scale I have to renew it manually, otherwise if the AutoRenewSessionLock kicks in, another instance may pick up the message or the same instance will start processing the same message again even though such is in progress.

Warning: This horse is dead 🐴!

You need to process your messages in FIFO style? Better use some other technology and stop waiting for Microsoft to support their own technology stack!

You are absolutely right. After hours of effort, I saw that the team was still in another realm.
This was not the first time. I hope it will be the last time.