Jeff,
While that will be useful, instance throttling is very much a blunt force instrument. We are currently looking at a resource based consumption throttler. So monitoring the use of each resource that gets used and has some sort of rate limit associated. Then we are building something that will handle that accordingly. I'll sit down and thrash out in detail whether we can get something working that would handle this nicely.
In particular the reason we are looking at this is that we are effectively partitioning our data across multiple rate limited pieces of storage, so in theory we'd like to avoid the situation where we are limiting based on our lowest throughput piece of storage.
Rather we'd like to circuit break those calls quickly and return a 429 or similar to the caller and continue to consume all of our available throughput on everything else.
At it's worst, the durable entities sound like a potentially better storage for this than the Redis cache we are using now.
I guess what I'm saying is that I suspect for most consumers, instance level throttling potentially won't cut it (also because a single instance is quite capable of completely overwhelming a downstream resource all on its own). So investigating a more granular level of throttling would be well worthwhile. For now, we are quite happy to continue investigating, but if you guys had some clever thoughts that might improve our direction that would be great.
Yes makes sense. Would be interested to learn more what you are thinking. Above the "blunt force" instance limiting we have been evaluating execution limiting, but what you are describe sounds even more granular than that. Almost something like "I have 400 locks for SQL, 2000 locks for Azure Storage -- hey functions, do your thing, but before you can run this line of code you need to make sure you have a lock first." Is that accurate?
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Jeff,
While that will be useful, instance throttling is very much a blunt force instrument. We are currently looking at a resource based consumption throttler. So monitoring the use of each resource that gets used and has some sort of rate limit associated. Then we are building something that will handle that accordingly. I'll sit down and thrash out in detail whether we can get something working that would handle this nicely.
In particular the reason we are looking at this is that we are effectively partitioning our data across multiple rate limited pieces of storage, so in theory we'd like to avoid the situation where we are limiting based on our lowest throughput piece of storage.
Rather we'd like to circuit break those calls quickly and return a 429 or similar to the caller and continue to consume all of our available throughput on everything else.
At it's worst, the durable entities sound like a potentially better storage for this than the Redis cache we are using now.
I guess what I'm saying is that I suspect for most consumers, instance level throttling potentially won't cut it (also because a single instance is quite capable of completely overwhelming a downstream resource all on its own). So investigating a more granular level of throttling would be well worthwhile. For now, we are quite happy to continue investigating, but if you guys had some clever thoughts that might improve our direction that would be great.
Yes makes sense. Would be interested to learn more what you are thinking. Above the "blunt force" instance limiting we have been evaluating execution limiting, but what you are describe sounds even more granular than that. Almost something like "I have 400 locks for SQL, 2000 locks for Azure Storage -- hey functions, do your thing, but before you can run this line of code you need to make sure you have a lock first." Is that accurate?